System and method for monitoring electronic communications

ABSTRACT

A method for monitoring mobile communication and generating alerts associated with targeted content. A monitored user&#39;s mobile communication device forwards an incoming or outgoing communication to a monitoring user&#39;s mobile communication device. The monitored user&#39;s or monitoring user&#39;s mobile communication device generates an alert if it is determined that the incoming or outgoing communication contains targeted content, and an alert is provided on the monitoring user&#39;s mobile communication device.

PRIORITY CLAIM AND REFERENCE TO RELATED APPLICATION

This application is a continuation-in-part of U.S. patent applicationSer. No. 16/194,748, filed Nov. 19, 2018, and incorporated in itsentirety by reference herein, which is a continuation of U.S. patentapplication Ser. No. 14/466,586, filed Aug. 22, 2014. U.S. patentapplication Ser. No. 14/466,586 is incorporated in its entirety byreference herein. This application also claims priority to and thebenefit of U.S. Provisional Patent Application Ser. No. 61/959,395,filed Aug. 22, 2013, entitled “METHODS AND SYSTEMS FOR MOBILECOMMUNICATION MONITORING AND ALERTS.” U.S. Provisional PatentApplication Ser. No. 61/959,395 is incorporated in its entirety byreference herein.

FIELD OF THE INVENTION

A field of the invention is mobile communications. Particularembodiments of the invention are applicable to monitoring electroniccommunications between mobile communication devices.

BACKGROUND OF THE INVENTION

Short Message Service (SMS) is a text messaging service component ofphone, web, or mobile communication systems, using standardizedcommunications protocols that allow the exchange of text messages(currently up to 160 characters for SMS, though this could change in thefuture) between fixed line, mobile phone devices or mobile communicationdevices. Current communication protocols used to deliver such servicesinclude GSM, CDMA, GPRS; 3G networks such as W-CDMA, EDGE or CDMA2000and 4G networks, though additional protocols and networks may beimplemented. Text messaging is a widely used communications serviceprovided by wireless carriers (e.g. U.S. Cellular), communicationcompanies (e.g. AT&T) and mobile app companies (e.g. Whatsapp, KiK).

Multimedia Messaging Service, or MIMS, is a standard way to sendmessages that include multimedia content (images, video, etc.) to andfrom mobile phones and mobile communication devices. It extends the coreSMS (Short Message Service) capability that currently allows exchange oftext messages only up to 160 characters in length. A popular use of MMSis to send photographs from camera-equipped handsets, although it isalso popular as a method of delivering news and entertainment contentincluding videos, pictures, text pages and ringtones. The standard isdeveloped by the Open Mobile Alliance (OMA), although during developmentit was part of the 3GPP and WAP groups.

Text messages, images (pictures), video and audio can be transmittedusing Wi-Fi, Bluetooth or other wireless technology over the Internetand without use of cellular communication infrastructure. As such, SMSand MIMS are emulated by mobile applications (“Mobile Apps”) thatprovide this type of functionality. Additional functionality such asaudio, video and audio/video content can be transmitted and receivedwith these types of Mobile Apps.

Though electronic messaging such as, but not limited to, SMS, MIMS andtext messaging (i.e. via mobile apps) has been widely adopted and hasproven very useful, the inherent dangers of electronic messaging andrelated communications such as cyber-bullying, the transmission andreceiving of sexual images (“sexting”) and other inappropriate or othertargeted content have been well publicized. The present inventor hasrecognized that the need to monitor these types of communications duringcell phone (more specifically, what is commonly referred to as a“smartphone”) or mobile communication device use is becomingincreasingly more important.

Other growing issues related to the private usage of mobilecommunications devices such as, but not limited to, laptop computers,tablet PCs (tablet computer), gaming devices, cell phones, PDAs(Personal Data Assistants) and other communication devices is withunsolicited contact by predators to children, facilitation of illegaldrug and alcohol activities, and the malicious broadcasting of contentintended to be private. Due to the portability of mobile communicationsdevices such as cell phones, children (and people) have regularopportunities to engage in electronic messaging such as text messageexchanges that have inappropriate or illegal content unbeknownst totheir parents.

Many parents and guardians that have children with mobile communicationdevices (e.g. cell phones or other devices that enable text messaging)are interested in protecting their children from the inherent dangersand ramifications associated with inappropriate or undesirable textmessaging communications described herein. However, the present inventorhas recognized that such parents or guardians are unable to easily oreffectively monitor the content being transmitted and received on theirchildren's cell phones or other devices that enable text messaging andelectronic communications. This is due in part to the lack oftransparency, anonymity and the mobile nature of cell phones and otherhand-held (mobile) communication devices.

Text messages, images, video and audio can also be transmitted andreceived directly over the Internet using a Wi-Fi, Bluetooth or viaother technologies that enables mobile communication devices to connectwirelessly to the Internet. To this end, mobile applications (mobileapps) now exist that emulate SMS and MIMS functionality.

In general, it has been recognized that users (e.g. parents, guardiansand family member) of mobile communication devices (for example but notlimited to SMS and MMS messages communicated via a handheld smartphonesuch as a BlackBerry, iPhone, cellular phone running an Androidoperating system or another portable computing device capable ofreceiving and/or sending wireless communications) face concernsregarding various aspects of communications to/from such mobilecommunication devices. Examples of this include the appropriateness,inappropriateness, safety and legality of such communications. Forexample, people (such as parents or guardians of children) may beconcerned about the language, the meaning of acronyms, the meaning ofshorthand, content, pictures and/or elements included in the childcommunications.

Some example systems and methods of the present invention address one ormore of these concerns by providing preferably near real-time monitoringof text messaging to address these and other related problems andissues.

SUMMARY OF THE INVENTION

An example embodiment of the invention provides, among other things, amethod for monitoring mobile communication and generating alertsassociated with targeted content. In an example method, a link isestablished between monitoring user's mobile communication device and amonitored user's mobile communication device to receive communicationsfor monitoring purposes. In some example embodiments, the monitoringuser's mobile communication device is authenticated to receivecommunications from a monitored user's mobile communication device(e.g., for monitoring purposes). The monitored user's mobilecommunication device forwards an incoming or outgoing electroniccommunication to the monitoring user's mobile communication device. Themonitored user's or monitoring user's mobile communication devicegenerates an alert if it is determined that the incoming or outgoingelectronic communication contains targeted content. The alert isprovided on the monitoring user's mobile communication device, e.g.,displayed, sounded, presented via haptic feedback, etc. In someembodiments, the targeted content associated with the alert and/or adescription of the target content (e.g. description of an acronym) maybe provided on the monitoring user's mobile communication device forreview by the monitoring user.

BRIEF DESCRIPTION OF THE DRAWINGS

The foregoing and other features, aspects and advantages of theinvention are described in detail below with reference to the drawingsof various embodiments, in which:

FIG. 1 is a schematic diagram of an embodiment for facilitating wirelesscommunication among communication devices, in accordance with someembodiments.

FIG. 2 shows a block diagram of a method for monitoring communicationson mobile communication devices and generating alerts in accordance withsome embodiments.

FIG. 3 is an example of a portion of a data structure for use inaccordance with some embodiments.

FIG. 4 is a depiction of an embodiment for a parent monitoring theirchild's/children's mobile communication device from their own mobilecommunication device by receiving copies of text messages coming intoand going out of the child's mobile communication device. Thisembodiment could be viewed as a message forwarding scheme. Morespecifically, the software (mobile app) forwards incoming and outgoingtext messages occurring on the child's mobile communication device tothe parent's mobile communication device.

FIG. 5 is a depiction of an embodiment for a parent monitoring theirchild's/children's mobile communication device by receiving copies oftext messages going out of the communication device. This embodimentcould be viewed as a message forwarding scheme. More specifically, thesoftware (mobile app) forwards outgoing text messages from the child'smobile communication device to the parent's mobile communication device.

FIG. 6 is a flowchart illustrating an example process for providingcopies of SMS communications from a mobile communication device beingmonitored to one or more mobile communication devices authenticated orregistered to monitor such communications in accordance with someembodiments.

FIG. 7 is a flowchart illustrating an example process for providingcopies of MIMS communications from a mobile communication device beingmonitored to one or more mobile communication devices authenticated orregistered to monitor such communications in accordance with someembodiments.

FIG. 8 is a flowchart illustrating an example process for providingcopies of audio, video or audio/video communications from a mobilecommunication device being monitored to one or more mobile communicationdevices authenticated or registered to monitor such communications inaccordance with some embodiments.

FIG. 9 is a flowchart illustrating an example process for providingcopies of electronic messages from a mobile communication device beingmonitored to one or more mobile communication devices authenticated orregistered to monitor such electronic messages in accordance with someembodiments.

FIG. 10 is a flowchart illustrating an example process for providingcopies of outgoing SMS, MMS or any electronic message from a mobilecommunication device being monitored to one or more mobile communicationdevices authenticated or registered to monitor such communications inaccordance with some embodiments.

FIG. 11 is a flowchart illustrating an example process for providingcopies of incoming SMS, MIMS or any electronic message from a mobilecommunication device being monitored to one or more mobile communicationdevices authenticated or registered to monitor such communications inaccordance with some embodiments.

FIG. 12 is a flowchart illustrating an example process for providingcopies of incoming SMS, MIMS or any electronic message communications,based on certain conditions being met, from a mobile communicationdevice being monitored to one or more mobile communication devicesauthenticated or registered to monitor such communications in accordancewith some embodiments.

FIG. 13 is a flowchart illustrating an example process for generatingalerts (warnings) for one or more mobile communication devicesauthenticated or registered to monitor communications on the mobilecommunication device being monitored when certain conditions have beenmet or target content has been matched to, based on predefineddata/content stored in a data structure or database (Target ElementData) on either the monitoring or monitored mobile communication devicein accordance with some embodiments.

FIG. 14 is a flowchart illustrating an example process for providingcopies of incoming audio communications, video communications oraudio/video communications based on certain conditions being met, targetcontent being matched to or target content having a close proximitymatch to predefined data/content stored in a data structure or databasefrom a mobile communication device being monitored to one or more mobilecommunication devices authenticated to monitor such communications inaccordance with some embodiments.

FIG. 15 is a flowchart illustrating an example process for providingcopies of all keystrokes or characters entered in or images created(i.e. pictures from camera feature on mobile device) on a mobilecommunication device being monitored by one or more mobile communicationdevices authenticated or registered to monitor such keystrokes andimages in accordance with some embodiments.

FIG. 16 is a schematic diagram of an embodiment for facilitatingwireless communication among communication devices, in accordance withsome embodiments, including communications through the Internet via aWi-Fi, Bluetooth or other wireless technology connection.

FIG. 17 is an example of a user interface that may be incorporated onthe monitoring cell phone, smartphone or mobile communication device toenable the monitoring user to view, in the case of firmware, textmessages or SMS that have occurred on the monitored user's cell phone,smartphone or mobile communication device, and are about to occur (pastand present).

FIG. 18 is an example of a user interface that may be incorporated onthe monitoring cell phone, smartphone or mobile communication device fordisplaying a text message or SMS that is associated with an alert (orwarning) that was generated by the software because it detected targetedcontent (e.g. possible inappropriate content) that occurred on themonitored user's cell phone, smartphone or mobile communication device.

FIG. 19 is an example of a hybrid peer-to-peer network architecture foruse in example methods.

FIG. 20 is another example of a hybrid peer-to-peer network architecturefor use in example methods.

FIG. 21 shows an example process performed by the monitored user'smobile communication device for analyzing incoming or outgoingcommunications on the monitored user's communication device, using an AImodel trained with targeted content.

FIG. 22 shows an example process for providing copies of obfuscated(e.g., blurred or distorted) image or video communications (e.g.,electronic messages) from a mobile communication device being monitoredto one or more mobile communication devices.

FIG. 23 shows an example process for determining if an image or videothat resides on a monitored user's mobile communication device hascharacters, words or sentences embedded into the image or video.

FIG. 24 shows an image/picture including embedded text for illustratingan example method.

FIG. 25 shows an example method for capturing, analyzing, andtransmitting an image or recording a video stream being presented, e.g.,displayed, on an output interface, e.g., a screen, of a mobilecommunication device.

FIG. 26 shows an example present-day check method for analyzing contentand transmitting the content to a monitoring user's mobile communicationdevice.

FIG. 27 shows example screenshots of a user interface of a monitoringuser's mobile communication device after receiving transmitted content.

FIG. 28 shows an example computer appliance for performing examplecontent monitoring methods.

DETAILED DESCRIPTION

Example embodiments of the invention provide, among other things,systems and methods for monitoring electronic communications. Suchsystems and methods in example embodiments further provide alerts ofparticular targeted content, such as inappropriate or illegal content,within electronic communications. Example systems and methods canprovided for monitoring a single user, or multiple users, and can beperformed by a single monitoring user or by multiple monitoring users.Non-limiting examples of mobile communication devices include cellphones, smartphones, iPhones, Windows phones, Samsung cell phones,Android phones, cell phones that have memory and processor(s), cellphones that run a mobile operating system such as Android, webOS, iOS,Blackberry OS, Symbian, Windows Mobile Professional, tablet computers,iPads, iPods, PDAs (Personal Digital Assistant) or other portablecommunication devices. Future mobile communication devices may includetechnologies under development such as Google Glass, smartphonesincorporating cloud-based technologies or cloud-based operating systemsor cloud-based computing schemes.

As used herein, the terms text message, text messages or text messagingis intended to include SMS (Short Message Service), MIMS (MultimediaMessaging Service) and other types of electronic messaging services orapplications including but not limited to those that transmit text, textstrings, characters and/or multimedia over the Internet via a Wi-Fi,Bluetooth or other wireless technology connection. Future messagingservices that utilize other electronic data transmission technologies,for example X-ray, RF (Radio Frequency), infrared, visible light,ultraviolet and gamma rays technologies or cloud computing technologiesor schemes (e.g. cloud messaging), may be applicable to some embodimentsherein.

Mobile communications devices include, but are not limited to, PersonalDigital Assistants (PDAs), cell phones, mobile phones, Smartphones,iPods, iPads, gaming devices, mobile computers, Tablet PCs and any otherelectronic device for which transmitting and receiving text messages arefrequently utilized (also referred to herein as a mobile communicationdevice). Example embodiments described herein refer to cell phones,however, such description is intended to also apply, where feasible, tomobile communication devices generally.

As used herein, the term software, mobile application, mobile app isintended to mean either software or firmware. Note that a mobile app maybe downloaded and installed on a smartphone or mobile communicationdevice whereby a smartphone or mobile communication device may havehardware or chips that have the software (application code) programmedinto them. In the case firmware, the software is essentially built intothe hardware of the smartphone or mobile communication device (whichdoes not need to be downloaded or installed). Additionally, it is alsopossible to incorporate software into the operating system of thesmartphone or mobile communication device so that it is integrated intothe smartphone or mobile communication device (not needing mobile appsoftware to be downloaded and installed by the end user). It will beappreciated that description herein for example methods will likewise beapplicable to suitably configured mobile communications devices,software or firmware (e.g., mobile apps), or networks or architectures(peer-to-peer or other networks) that perform one or more features ofthe method.

As used herein, the term “electronic message” or the term “electroniccommunication” is intended to mean any type of message that can betransmitted or received by a mobile communication device, including butnot limited to a text message, SMS, image (picture), MMS, audio clip,video clip, audio/video clip, live audio segment, live video segment orelectronic message.

In an example method for monitoring mobile communication and generatingalerts associated with targeted content, such as potentiallyinappropriate or illegal content, a link is established between a user'smobile communication device to be monitored and a user's mobilecommunication device to receive communications for monitoring purposes.The devices may also be authenticated or registered, though suchauthentication is not required in all embodiments. The monitoring user's(also referred to herein as monitoring party or monitoring party's)mobile communication device (“monitoring device”) receives communication(i.e. via message forwarding), either incoming or outgoing, from auser's (also referred to herein as monitored party or monitored party's)mobile communication device being monitored (monitored device). In someexample embodiments, only incoming, or alternatively only outgoing,communications are monitored.

The monitored user's mobile communication device generates and transmitsan alert to the user's mobile communication device monitoring thecommunications if the application software running on the monitoreduser's mobile communication device determines that a communicationcontains targeted content. Determining whether a communication containstargeted content can be accomplished, for example, by comparing thecontent of the communication to a library of targeted content, such asinappropriate and/or illegal violations, stored in a local database (ordata structure) including text strings, images, audio clips and videoclips. Example software can search for exact matches or comparablematches (e.g. comparable match could be an image in the local databasethat resembles the image in a communication) within the local databaseto determine if it should generate an alert to be transmitted to themonitoring user's mobile communication device. The alert is provided,e.g., displayed, sounded, provided via haptic feedback, etc. on themonitoring mobile communication device.

For example, the monitoring user's mobile communication device canreceive alerts transmitted (forwarded) from a monitored user's mobilecommunication device when potentially inappropriate or illegal contentis determined by the software after searching a local database (or datastructure) of predefined text strings and/or images and/or audio clipsand/or video clips residing on the user's mobile communication devicebeing monitored.

In a particular example method, the communications between the monitoreduser and the monitoring user is implemented in a peer-to-peer networkscheme (i.e. peer-to-peer architecture). For example, run-time softwareof a mobile application resides (is installed) on all monitoring andmonitored mobile communication devices. The monitored mobilecommunication device's run-time software forwards text messages, MMS orother electronic communications to the monitoring mobile communicationsdevice's run-time software and not to an intermediary server. In thisway, an example implementation of some embodiments need not require anintermediary or centralized server (or servers) to carry out itsrun-time functionality. Servers may be implemented for downloadingexample software (e.g., mobile app) to mobile communication devices anduploading keywords and/or images and/or audio clips to the localdatabase or data structure on the monitored user's mobile communicationdevice (non-run-time activities). However, during the run-time executionof the software providing the communication monitoring and alerts, nointermediary or centralized servers are necessary by the exampleapplication software to provide this functionality.

It should be noted that the mobile communications service providers(i.e. Verizon, AT&T, T-Mobile, etc.) may use servers to implement theirspecific services for SMS, MIMS, voice mail, live voice communication orinstant messaging (IM), however this level of communication for anexample implementation of the invention is preferably carried out as arequest from the application software to the operating system of themobile communication device, which then requests service to receive aninterrupt pertaining to a new incoming text message or outgoing textmessage, from the mobile communication device's operating system.Another example implementation is to incorporate polling to look for newactivity occurring in the communication buffers for text messages, MIMSor other types of electronic communications. Mobile network (cellularnetwork) providers include services for SMS and MMS by incorporatingSMSC (Short Messaging Service Center) and/or MSC (Multimedia MessagingService Center) in there network elements.

Another embodiment provides, among other things, a method for monitoringmobile communication including reading and storing each character inputby a monitored user when entering in information on a monitored mobilecommunication device (e.g. keyboard/keyboard device, on-screen keyboard,voice input, pointing device). The character inputs are analyzed and/orparsed for a text message style communication that utilizes the Internetfor a transmission network (rather than a SMSC or MMSC type network).The content of the text strings (i.e. contiguous characters stored thatwhere input by the user) stored by the software that is intended foranother user to read or see (e.g. text message, images) is transmittedto the monitoring user's mobile communication device.

In an example embodiment, the monitored user's mobile communicationdevice receives characters input on the monitored user's mobilecommunication device and analyzes/parses the character strings for textmessage style communications (the portion of the characters thatcomprise a message or image to be read/viewed by another user) to becompared against a list of predefined text strings, images and videoclips that are deemed to be targeted content, e.g., inappropriate orillegal content, for a direct match or a comparable match. Provided adirect match or comparable match is determined, then the characterstring or image that may have the target content is transmitted(forwarded) to the monitoring user's mobile communication device. Inanother example embodiment, provided a direct match or comparable matchis determined, the example software generates an alert that istransmitted (sent) to the monitoring user's mobile communication device.

In an example embodiment, the information about the names of theapplications (mobile app) running on the monitored mobile communicationdevice can be read from the device's operating system and used by thesoftware as intelligence as to what text messaging type software(Whatsapp, KiK, Snapchat) is being used by the monitored user. Based onthis information, the format of the character strings and images beingcommunicated to other mobile communication devices can be more readilydetermined. For example, Snapchat allows multimedia (for example,pictures) to be transmitted from one party's smartphone to anotherparty's smartphone whereby the user's receiving the pictures on theirsmartphone will only be able to view them for a designated period oftime and then they are no long accessible. This embodiment would, forexample, enable a monitoring user to receive a copy of and store suchpictures for viewing at their convenience.

An example system and method works with two or more mobile communicationdevices (particular non-limiting examples including IPHONE, BLACKBERRY,SAMSUNG GALAXY). At least one mobile communication device is the devicebeing monitored (“Monitored Device”) and at least one additional mobilecommunication device is the device conducting the monitoring(“Monitoring Device”). The monitoring and monitored mobile communicationdevices all have application software (e.g., an app) downloaded,installed and running on the devices in order to implement thefunctionality. In addition, data is stored in a database or datastructures on the monitored mobile communication device for supportingthe application software.

An example application software for monitoring mobile communicationsprovides various functions, such as:

1) the forwarding of communications on the monitored mobilecommunication device(s) to the monitoring mobile communicationdevice(s). It should be noted that a monitored mobile communicationdevice can be monitored by more than one monitoring mobile communicationdevices and that a monitoring mobile communication device can monitormultiple monitored mobile communication devices;

2) the generation and transmission of alerts when the applicationsoftware on a monitored mobile communication device determines that acommunication has potentially inappropriate, illegal or target content.This function preferably utilizes a database or data structure ofpredefined text strings, images, audio clips and video clips stored onthe monitored mobile communication device (or on the monitoring mobilecommunications device);

3) the forwarding of characters or images generated by the monitoredmobile communication device as the user types (or otherwise enters in)characters, one by one, on a character input mechanism, an example forwhich is an on-screen keyboard or a physical keyboard, to the monitoringmobile communication device(s) or as the user creates images facilitatedby the mobile communication device, an example for which is a picturetaken using a camera integrated into a mobile phone. Voice recognitioninput mechanisms is another example of an input mechanism to a mobilecommunication device.

Additional embodiments provide, among other things, a method formonitoring mobile communication by reading and storing each image movedfrom memory to the image buffer of the mobile communication device (i.e.the operating system of the mobile communication device) by a monitoreduser (e.g. a picture/photograph to be sent to another user) to be storedby the software and transmitted (forwarded) to the monitoring user'smobile communication device. In this example method, the monitoreduser's mobile communication device moves an image from memory to theimage buffer of the mobile communication device and compares that imageto list of predefined images that are deemed to be targeted content,such as inappropriate, illegal or targeted content, for a direct matchor a comparable match. Provided a direct match or comparable match isdetermined, then the image that may have the targeted content istransmitted (forwarded) to the monitoring user's mobile communicationdevice. Alternatively or additionally, the monitored user's mobilecommunication device moves an image from memory to the image buffer ofthe mobile communication device and compares that image to list ofpredefined images that are deemed to be targeted content for a directmatch or a comparable match. Provided a direct match or comparable matchis determined, then the example software generates an alert that istransmitted (sent) to the monitoring user's mobile communication device.

Example embodiments provide methods and systems for monitoring andanalyzing communications of a monitored user on behalf of a monitoringuser, for transmitting communications (message forwarding) to monitoringusers. Additionally, alerts of potentially inappropriate, illegal ortargeted content are provided (transmitted) to the monitoring users. Forexample, text messages, SMS messages, MMS messages, IMs, e-mails, socialnetwork site postings or voice mails of a child may be monitored onbehalf of a parent. In one embodiment, communications occurring on achild's communication device (cell phone, smartphone, iPod, iPad, etc.)are forwarded (transmitted) to one or more monitoring user(s).

Additionally, an algorithm can be used to analyze the communication formatches (or close similarities) to stored elements (for example, words,text strings, character strings, acronyms, images, audio clips, videoclips) (“Stored Elements”) in a database residing on the communicationdevice being monitored, through a comparison process to assess thepotential of targeted content, such as inappropriate or illegal content.The Stored Elements preferably are updated and uploaded periodically tomaintain up-to-date information in the local database (note that this isan operation that need not be executed as part of some embodiments'run-time functionality, e.g., the upload does not negate an example peerto peer scheme). One reason for the updates is to stay current with thelatest acronyms and shorthand used during text messaging (SMS messages)by children. New acronyms with potentially inappropriate or illegalcontent are created by users frequently. For example, currently theacronym GNOC has been known to mean “Get Naked On Camera”. When thealgorithm determines that the communication has a match or closesimilarity to a match, it will generate an alert that is transmitted(sent) to the monitoring user's communication device. In anotherexample, when comparing an image from the monitored user's mobilecommunication device to the Stored Elements, the comparison process mayuse a close proximity match to determine if the image contains sexualcontent. Although there may not be a direct match, it can be determinedthat through a close proximity comparison algorithm, for example, thatan image should be categorized as sexual in its nature.

Although there are many applications for different embodiments, oneexample application enables parents to monitor text messages from theircell phone, smartphone or mobile communication device occurring on theirchildren's cell phones, smartphones or other mobile communicationsdevices. This includes monitoring outgoing messages (e.g. text messages)sent by the child and incoming messages (e.g. text messages) beingtransmitted to the child's cell phone (or communication device) byforwarding the messages to the parent's cell phone, smartphone or mobilecommunication device. The monitoring (or the viewing of these messages)occurs on the parent's cell phone, smartphone or mobile communicationdevice through a mobile app that is installed on the parent's device andchild's device.

In one embodiment, for security purposes, authentication (orregistration) may be performed between the monitoring device and themonitored device (“remote cell phone”) to be monitored, prior to anytext messages or electronic communications being forwarded (transmitted)to the monitoring device. In one embodiment, this is accomplishedthrough a request sent by the monitoring device (or devices) to a cellphone, smartphone or mobile communication device to be monitored(monitored device), e.g., a remote cell phone. The person with theremote cell phone must accept this request to validate the monitoringparty's cell phone, smartphone or mobile communication device andacknowledge that the monitoring party will have the capability tomonitor text messages and other electronic communications occurring ontheir cell phone, smartphone or mobile communication device.

Preferred embodiments will now be discussed with respect to thedrawings. The drawings include schematic figures that are not to scale,which will be fully understood by skilled artisans with reference to theaccompanying description. Features may be exaggerated for purposes ofillustration. From the preferred embodiments, artisans will recognizeadditional features and broader aspects of the invention.

Although several embodiments, examples and illustrations are disclosedbelow, it will be understood by those of ordinary skill in the art thatthe invention(s) described herein extend(s) beyond the specificallydisclosed embodiments, examples and illustrations and includes otheruses of the invention(s) and obvious modifications and equivalentsthereof. Embodiments of the invention(s) are described with reference tothe accompanying figures. The terminology used in the descriptionpresented herein is not intended to be interpreted in any limited orrestrictive manner simply because it is being used in conjunction with adetailed description of certain specific embodiments of theinvention(s). In addition, embodiments of the invention(s) can compriseseveral novel features and it is possible that no single feature issolely responsible for its desirable attributes or is essential topracticing the invention(s) herein described.

Certain aspects, advantages, and novel features are described herein. Itis to be understood that not necessarily all such advantages may beachieved in accordance with any particular embodiment of the invention.Thus, for example, those skilled in the art will recognize that theinvention may be embodied or carried out in a manner that achieves oneadvantage or group of advantages as taught herein without necessarilyachieving other advantages as may be taught or suggested herein.

FIG. 1 is a schematic diagram of an embodiment of a system forfacilitating wireless communication among communication devices, inaccordance with some embodiments. FIG. 1 shows an example of software(mobile app) communicating directly to other mobile communicationdevices in a peer to peer scheme and utilizing a mobile wirelessprovider's SMS/MMS infrastructure (i.e. cellular network). Thisillustrates a mobile wireless communication provider's infrastructureutilized to support some embodiments. FIG. 1 further shows an example ofapplication level software (mobile app) according to particularembodiments. In FIG. 1, arrow with solid line indicates messages beingtransmitted to the monitoring party in a peer to peer scheme at theapplication level. Alerts or warnings are also transmitted to themonitoring party's mobile communication device in a peer to peer schemewhereby the software, data and databases necessary to carry out therun-time functionality of some embodiments reside on the monitoring andmonitored mobile communication devices and do not require logging intoan intermediary server to access the information provided by suchembodiments

FIG. 2 shows a block diagram representing a method for monitoring ofcommunications on mobile communication devices in accordance with someembodiments. In FIG. 2, communications are obtained from one or moremobile communication devices being monitored (a “Monitored Device”) andsuch communications are forwarded to one or more other mobilecommunication devices authenticated to monitor such communications (a“Monitoring Device”). Additionally, should the communications bedetermined to contain targeted content, such as potentiallyinappropriate or illegal content, than an alert will also be transmittedto the mobile communication devices authenticated to monitor theMonitored Device.

Before or concurrently with monitoring, a link is preferably establishedbetween the monitoring user's mobile communication device and themonitored user's mobile communication device. This link can beestablished, for instance by the software requesting and receiving amonitored user's mobile communication devices (e.g., ID number, phonenumber, etc.), sending and receiving (e.g., accepting) a message fromone or more of the devices to another one or more of the devices,handshaking, etc. Establishing the links can be accompanied byauthentication or registration in some embodiments. Such links can bepeer to peer or other direct link, server based, established duringinstallation for one or both of the devices, or in other manners. Suchestablishing of a link can be, but need not be, performed prior to orconcurrently with any of the example methods provided herein.

FIG. 3 shows an example portion of a data structure with stored textstrings, images, audio clips and video clips for the generation ofalerts. The example data structure may be in a database with a library(dictionary) of text strings, images, video clip and audio clips to beused when the software (mobile app) of an embodiment of this inventionanalyzes communications (e.g. SMS, MMS messages, voice mail) fortargeted content, such as inappropriate or illegal content, for thegeneration of alerts to be transmitted from the mobile communicationdevice. The data structure can be part of a database (or standalone) forwhich some embodiments use as a reference to determine, through directmatch or analyzed for a close proximity match, if communications aredeemed to be potentially inappropriate or illegal and warrant an alert(notice or warning) to be generated and transmitted to monitoringparties (e.g. an alert sent to a parent's mobile communication devicewhen potentially inappropriate content was detected on their child'smobile communication device). Example alerts that can be providedinclude but are not limited to, one or more of the following: icon onthe status bar of a monitoring mobile communication device's userinterface/main screen, icon displayed at any designated location on amonitoring mobile communication device user interface, pop up message,text message, SMS, MIMS, image, audio clip, live audio message, videoclip, live video message, audio/video clip, live audio/video message,vibration, instant message or any type of notification that can beincorporated with a mobile communication device (of a monitoring user).

An example of a close proximity match is determining if an image in acommunication (e.g. MIMS) matches that of an image in the datastructure/database that contains pre-defined text strings, images, audioclips and video clips of potential inappropriate or illegal content.Content analysis may incorporate image comparison, matrix comparison,raster comparison, pixel comparison, facial recognitionalgorithms/software/services or any other technique, software, utility,service or computational model used to determine that an image is aclose match to a defined or target content (PerceptualDiff utility,currently available via the Internet at pdiff.sourceforge.net, is aparticular, non-limiting example).

A specific, non-limiting example application monitors a MMScommunication that has a picture of a female breast. The MMScommunication is compared to similar images in the data structure ordatabase for a reasonable match (close proximity match). If the softwaredeems there is a reasonable match, then an alert may be generated andtransmitted, for example, to the monitoring parties (e.g. parent)indicating that an electronic message with potential sexual content wasdetected. The concept of children engaging in the transmission/receiptof sexually explicit images on smartphones (“Sexting”) activities hasbecome an issue with mobile communications, and such activity between aminor and an adult is currently illegal in the United States.

Preferably, example systems and methods accomplish their run-timefunctionality for providing monitoring capabilities and alerts through apeer to peer network scheme (i.e. peer to peer network architecture).Advantages to the peer to peer network scheme for this exampleapplication include but are not limited to the following:

1) Ease of use—There is no need to login to a server through aweb-browser on a cell phone, smartphone, mobile communication device,computer, PC, tablet PC or any other computing device that can browsethe Internet in order to access text messages being monitored. Exampleembodiments use application software (mobile app) that runs (isexecuted) on the monitoring and monitored cell phones hardware directlyand receives and stores the text messages being monitored in the memory(database) of the monitoring cell phone, smartphone or mobilecommunication device hardware directly and not on an intermediary orcentralized server.

2) Better data security for the text messages being monitored—Being thatall the text messages being forwarded (transmitted) to the monitoringcell phone, smartphone or mobile communication device are storedexclusively on the monitoring party's cell phone, smartphone or mobilecommunication device and not on a remote server or intermediary server,the potential for a security breach from an outside party is greatlyreduced.

3) Real-time monitoring—This technology enables the monitoring party toreceive the text messages and alerts on their cell phone, smartphone ormobile communication device in near real-time (approximately at the timethey occur on the child's cell phone, smartphone or mobile communicationdevice) and without any additional latency (delays) introduced by aserver-based architecture (i.e. two-tier architecture, client/serverarchitecture).

In other words, no servers are needed to display the text messagesprovided by the real-time text message monitoring feature and alerts ofsuch example run-time embodiments. Servers may be implemented to uploadthe application software (mobile app) or data to a mobile communicationdevice, For example, uploading the most recent, up-to-date list of textstrings, images, audio clips and video clips to the data structure ordatabase containing targeted content. Servers may be implemented byservices utilized, for which a specific non-limiting example is servicesof a cellular communications service provider or also known as wirelesscommunications service provider (FIG. 1).

To further clarify this point, the application software, data, databasesand signaling/control information for implementing the run-time aspectof such example embodiments is self-contained on the cell phones,smartphones or mobile communication devices themselves and does not relyon an external server to facilitate or provide any aspect of itsspecific application functionality.

The example application software only needs to reside on at least onemonitored mobile communication device (cell phone) and one monitoringmobile communication device (cell phone) (in any suitable distribution)in order to provide the functionality of this technology, though it canreside on multiple devices if desired. In other words, the technology'sapplication software preferably resides on a minimum of one cell phone(displaying the text messages, pictures, electronic communications andalerts, being monitored) and a minimum of one cell phone whose textmessages are being monitored and forwarded to the cell phone that isdoing the monitoring. The near real-time monitoring function in suchexample embodiments is implemented as a peer to peer network scheme(architecture) and does not require an intermediary server (e.g.application server) for executing its functionality for monitoringelectronic communications (e.g. text messages) or providing alerts. Inthis context, the example application software operates in a peer topeer network scheme implementing direct node to node communication whenissuing a command to transmit (forward) text messages or electroniccommunications from the monitored cell phone (mobile communicationdevice) to the monitoring cell phone (mobile communication device) incontrast to a server-based network scheme whereby one or more cellphones (mobile communication devices) communicate with or transmit textmessages or electronic communications to a computer server (that is notitself a cell phone, smartphone or mobile communication device).However, at least some embodiments may adopt some server basedfunctionality.

FIG. 4 shows a depiction of near real-time text message or SMS messagemonitoring for cell phones, smartphones or mobile communication devices.One example application may be a parent monitoring theirchild's/children's mobile communication device whereby the parentreceives copies of text messages or SMS messages coming into (beingreceived by) and going out (being transmitted out of) of the child'smobile communication device. Such an embodiment may be viewed as amessage forwarding scheme.

Another way to view this is that the software (mobile app) in thisembodiment forwards incoming and outgoing text messages or SMS messagesfrom the child's mobile communication device (“monitored device”) to theparent's mobile communication device (“monitoring device”). All textmessages (incoming & outgoing) occurring on the child's cell phone(mobile communication device) are relayed (forwarded) to the parent'scell phone (mobile communication device) by the example mobile app. Thedata (e.g., text messages to be monitored) preferably are stored on thechild's and parent's cell phone and not on an intermediary orcentralized server. The mobile app preferably is installed on theparent's cell phone (mobile communication device) and the child's cellphone (mobile communication device) and contains the execution code(i.e. software), database(s), libraries and data (including textmessages and electronic messages) to perform all operations necessary tocarry out the functionality of example methods.

Although not illustrated by FIG. 4, monitoring of more than one cellphone (mobile communication device) to be monitored can be done from thesame mobile app. Additionally, more than one monitoring party canmonitor the same child or children. For security purposes, optionalauthentication or registration may be performed on the monitoringparty's cell phone or to register the monitoring party's cell phone tothe monitored party's cell phone prior to any text messages beingrelayed (sent).

Although the above embodiment provides distinct advantages for manyapplications as discussed, in some other applications use of a cloudcomputing, grid computing, autonomic computing configuration or othertypes of network or computing schemes may be useful.

FIG. 5 is a depiction of an embodiment for a parent monitoring theirchild's/children's mobile communication device by receiving copies ofoutgoing text messages or SMS being transmitted out of the communicationdevice. This embodiment could be viewed as a message forwarding scheme.More specifically, the example software (mobile app) forwards outgoingtext messages or SMS messages from the child's mobile communicationdevice to the parent's mobile communication device. FIG. 5 illustratesnear real-time SMS message (text message) monitoring for cell phones(e.g., “smartphones,” or other mobile communication devices) of outgoingSMS messages (text messages), implemented as a peer-to-peer networkarchitecture. All outgoing text messages occurring on the child's cellphone are relayed (sent) to the parent's cell phone by the mobile app.The data (text messages to be monitored) preferably are storedexclusively on the child's and parent's cell phone and not on anintermediary or centralized server. The example mobile app is installedon the parent's cell phone and the child's cell phone and contains theexecution code, database, libraries and data (including text messages orelectronic messages) to perform all operations necessary to carry outthe functionality of some embodiment. Although not illustrated by thisdiagram, monitoring of more than one party's mobile communication devicecan be performed from a single monitoring mobile communication device.Additionally, more than one monitoring party's mobile communicationdevice can monitor the same child's or children's mobile communicationdevice (monitored mobile communication device). For security purposes,optional authentication or registration may be performed on themonitoring party's cell phone prior to any text messages or electronicmessages being forwarded (transmitted).

Various example monitoring methods are shown in FIGS. 6-15. Forinstance, one particular example use for this technology, as a mobileapplication (mobile app), is to enable a parent to monitor the textmessages occurring on their children's cell phones from their personalcell phone (e.g., FIG. 4). However, in addition to text messagemonitoring, this can enable the monitoring of other activities orfeatures that can occur on a child's cell phone including but notlimited to the monitoring of incoming and outgoing MMS messages (FIG.7), images, audio messages, video messages, audio/video messages (FIG.8) and other types of electronic messages (FIG. 9).

Other examples of monitoring functionality that this technology enableson cell phones, smartphones or mobile communication devices include, butare not limited to, cell phone conversations (voice phone calls)monitoring (monitoring cell phone numbers called/received), websites(URLs) visited, applications installed or uninstalled on a remotedevice, the turning on/off of the GPS or other location detectingtechnologies, features or services that may exist today or in the futureon a remote cell phone (or communication device) to be used inidentifying the location of the remote cell phone or communicationdevice (e.g. to find the location of a child), other tracking featuresthat can be used to proactively alert parents when a child travelsbeyond certain geographical boundaries or predefined limits related togeographical position and any or all statistics or metrics related tothe aforementioned functions or any other functions that can bemonitored on a cell phone, smartphone or mobile communication device.Some examples of statistics include: the number of text messagesreceived from a particular contact, the number of text messages sent toa particular contact, which contacts communicated to the most to thechild's cell phone (smartphone), the number of new contacts made duringa particular timeframe, how many times a child left and entered aparticular geographic zone (i.e. residence) and the associated dates andtimes, the number of times a particular website was visited, the numberof new websites visited in a particular timeframe, and others.

Particular example embodiments provide the near real-time monitoring oftext messages, SMS or MIMS messages from a third party cell phone (themonitoring cell phone) for all incoming and outgoing text messages, SMSor MIMS messages occurring on a designated cell phone to be monitored(the remote cell phone) and any other mobile communication devices suchas a smartphone, PDA, tablet PC or computer that can transmit andreceive text messages, electronic messages, SMS or MMS messages viacellular technology, Internet technology or other types of datacommunication technologies that enable computers or mobile communicationdevices to communicate and share data existing today or in the future,as shown in FIGS. 4, 6, and 7.

FIG. 6 is a flowchart illustrating an example process for providingcopies of SMS or text message communications from a mobile communicationdevice being monitored to one or more mobile communication devicesoptionally authenticated to monitor such communications in accordancewith some embodiments.

FIG. 7 is a flowchart illustrating an example process for providingcopies of MMS communications or images from a mobile communicationdevice being monitored to one or more mobile communication devicesoptionally authenticated to monitor such communications in accordancewith some embodiments.

FIG. 8 is a flowchart illustrating an example process for providingcopies of audio, video or audio/video communications from a mobilecommunication device being monitored to one or more mobile communicationdevices optionally authenticated to monitor such communications inaccordance with some embodiments.

FIG. 9 is a flowchart illustrating an example process for providingcopies of any electronic message from a mobile communication devicebeing monitored to one or more mobile communication devices optionallyauthenticated to monitor such electronic messages in accordance withsome embodiments.

Another embodiment provides a method for transmitting only the incomingtext messages, SMS, MIMS messages or any type of electronic messagecommunication occurring on one or more mobile communication devicesbeing monitored (e.g. child's cell phone) to a monitoring mobilecommunication device (FIG. 11). One example use would be for a parent tobe able to monitor from their smartphone, text messages their child isreceiving on their smartphone from one or more parties. FIG. 11 is aflowchart illustrating an example process for providing copies ofincoming text messages, SMS, MMS or any electronic messagecommunications from a mobile communication device being monitored to oneor more mobile communication devices optionally authenticated to monitorsuch communications in accordance with some embodiments. FIG. 10 is aflowchart illustrating an example process for providing copies ofoutgoing text messages, SMS, MIMS or any electronic messagecommunication from a mobile communication device being monitored to oneor more mobile communication devices optionally authenticated to monitorsuch communications in accordance with some embodiments. One example usewould be for a parent to be able to monitor from their smartphone, textmessages their child is transmitting on their smartphone to one or moreparties.

Another embodiment of this technology provides a method for transmittingonly the incoming and/or outgoing text messages, SMS, MIMS, images,audio clips, video clips, audio/video clips or electronic messaging froma mobile communication device being monitored to monitoring mobilecommunication devices that have/include a word, text string, image,image snippet, audio clip, video clip or other identifier that has beenpredefined or is target content (FIG. 12). FIG. 12 is a flowchartillustrating an example process for providing copies of text messages,SMS, MIMS, images, audio clips, video clips or any electronic messagecommunications, based on certain conditions being met, from a mobilecommunication device being monitored to one or more mobile communicationdevices optionally authenticated to monitor such communications inaccordance with some embodiments. One example use of this embodiment isto enable a parent to received copies of text messages, SMS, MIMS,images, audio, video or other electronic messages that match or have aclose proximity match to targeted words, text strings, acronyms, shorthand, codes, images, image snippets, audio, video or other identifiers.This allows a parent to judiciously monitor their children's electronicmessaging content only when there are reasonable grounds or probablycause.

Alerts can optionally be generated and displayed on the MonitoringParty's mobile communication device (cell phone) indicating that a matchor close proximity match was detected based on the comparison of thecommunication content to the predefined words, text strings, acronyms,short hand, codes, images, image snippets, audio, video, audio/video orother identifiers (FIG. 13). FIG. 13 is a flowchart illustrating anexample process for transmitting alerts (warnings) to one or more mobilecommunication devices optionally authenticated to monitor communicationson the mobile communication device being monitored when certainconditions have been met based on predefined data stored (StoredElements) or targeted content has been matched or reasonably matched(close proximity match) in a local data structure or database on themobile communication device being monitored or the monitoring mobilecommunication device, which is in accordance with some embodiments.

Another embodiment of this technology provides a method for transmittingincoming and/or outgoing voice/audio content or clips that match orreasonably match (close proximity match) a predefined audio content/clip(targeted content) stored in a library or database (FIG. 14). FIG. 14 isa flowchart illustrating an example process for providing copies ofincoming audio communications (message) or video communications(message), based on certain conditions being met, from a mobilecommunication device being monitored to one or more mobile communicationdevices optionally authenticated to monitor such communications inaccordance with some embodiments.

One example use of this embodiment is to enable a parent to monitorvoice/audio calls for selected words, sounds or other audio identifiers.This allows a parent to monitor their children's phone (audio)conversations by receiving specific audio clips, words, sounds, or otheraudio identifiers that match those that were predefined, when they occuron their children's cell phone(s) or mobile communication device. Alertscan also be sent to the monitoring party's cell phone or mobilecommunication device indicating that a match or close proximity matchhas occurred.

FIG. 15 is a flowchart illustrating an example process for providingcopies of all keystrokes or characters entered in (e.g., text message),images, audio, video, audio/video or multimedia message created (e.g.pictures from camera feature on mobile a device) on a mobilecommunication device being monitored by one or more mobile communicationdevices optionally authenticated to monitor such keystrokes, characters,images, audio content, video content, audio/video content or multimediamessages in accordance with some embodiments.

FIG. 16 is a schematic diagram of an embodiment for facilitatingwireless communication among communication devices, in accordance withsome embodiments. In particular with this example, communications can bethrough the Internet via a Wi-Fi, Bluetooth or other wireless technologyconnection.

FIG. 17 is an example of a user interface that may be incorporated onthe monitoring cell phone, smartphone or other mobile communicationdevice to enable the monitoring user to view, in the case of firmware,text messages or SMS that have occurred on the monitored user's cellphone, smartphone or mobile communication device, and are about to occur(past and present). That is, if a monitoring user opened this userinterface on their mobile communication device on a specific day, a copyor log of all text messages, for example, from the previous day'scommunication on the monitored user mobile communication device would beaccessible by scrolling backwards on the interface. Additionally andfollowing this logic, other past day's text messages or SMS (2 daysprevious, 3 days previous and so on) would be accessible by themonitoring user on this user interface. In another instance of thisexample, if a new text message or SMS is generated by the monitored userwhile the monitoring user is on/viewing this example user interface,then the new text message or SMS would be displayed in near real-time onthe user interface (in chronological order with the other text messagesor SMS). Therefore, the user would see the text message or SMS appear onthe user interface in near real-time as it was generated on themonitored user's cell phone, smartphone or mobile communication device.

A text message or SMS displayed on this example user interface couldoptionally include all information that a typical text message or SMSwould make available to software or a mobile application (mobile app)such as time and date stamp of when it was received or transmittedon/from the monitored user interface, the phone number or useridentification number of the monitored user's or one or more 3^(rd)party cell phones, smartphones or mobile communication devices involvedin the communication and any other information pertinent or availablefrom the message, from the messaging service providing the text messageor SMS or information related to the message. Additionally, this exampleuser interface would allow a pre-determined total number of textmessages or SMS to be stored for display (viewing may be implemented byenabling scrolling backwards and forward on the user interface) and inthis example, the software would either rewrite over the oldest messagewhen the storage limit was achieved by implementing a circular buffertechnique or would generate a warning message to the monitoring user'scell phone, smartphone or mobile communication device indicating thatthe text message or SMS log was full and that it should be cleared orindividual messages of the choice of the user should be deleted to makeroom for new, incoming text messages or SMS.

FIG. 18 is an example of a user interface that may be incorporated onthe monitoring cell phone, smartphone or mobile communication device fordisplaying a text message or SMS that is associated with an alert (orwarning) that was generated by the software because it detected targetedcontent (e.g. possible inappropriate content) that occurred on themonitored user's cell phone, smartphone or mobile communication device.In this example, the alert causes at least 2 actions to occur on themonitoring user's cell phone, smartphone or mobile communicationdevice. 1) It enables the monitoring user to open up or invoke a userinterface, like the example user interface (FIG. 18), that will displaythe text message or SMS that the software matched or determined was aclose proximity match to target content stored in a local database oneither the monitored user's cell phone, smartphone or other mobilecommunication device or on the monitoring user's cell phone, smartphoneor other mobile communication device; 2) The software will display analert that would attempt to capture the attention of the monitoring userto make he/she aware that a warning (alert) has occurred. This alertcould be displayed, for example, as an icon on the status bar of theprimary user interface of a cell phone, smartphone or other mobilecommunication device (FIG. 18) or it could be in the form of a textmessage, pop up message, audio message, image message, instant messageor other type of message that would attempt to gain the attention of themonitoring user when they looked at the display of or used their cellphone, smartphone or other mobile communication device.

FIGS. 19-20 illustrate features of peer-to-peer network architecturesthat can provide a network architecture for example monitoring methodsdisclosed herein. Generally, hybrid peer-to-peer networks can combineeither some or all the functionality of both peer-to-peer andclient/server network architectures. In FIGS. 19-20, monitored andmonitoring mobile communication devices, shown as Child and Parentdevices, as a nonlimiting example, provide two or more nodes. In FIG.19, the server acts as a communication bridge between nodescommunicating in a peer-to-peer fashion. The server provides theauthentication between child and parent nodes and facilitates acommunication channel between them in an example type of hybridpeer-to-peer network. In this example, the server does not act as acentralized server that provides most of the resources (e.g. storing andarchiving data, software, etc.) but acts as a server to help peers findone another and facilitate communications between them. FIG. 19 denotesan authenticated communication channel between parent and child nodeswith the type of connecting line used (solid, dashed, dot-dashed). FIG.20 illustrates another example peer-to-peer network structure wherebyevery node (though not required in all such structures) can communicatewith each other directly without the need of a server.

In an example hybrid-peer-to-peer network architecture, a linkestablishing communication channels for authenticated connections isprovided via a server (or one or more connected servers), e.g., an indexserver, or via the Cloud, e.g., an index server or other server in theCloud. The link manages the connections to facilitate unidirectional orbidirectional data transfers between the monitored and monitoringcommunication devices (nodes). In FIGS. 19-20 matching lines (e.g.,solid, dashed, dashed dot) depict authenticated communication channelsthat are facilitated by the server or Cloud link.

In some example hybrid peer-to-peer architecture embodiments, a server(e.g., centralized server) or Cloud server helps peers find each otherto provide monitoring and/or forwarding applications. Other exampleembodiments provide a hybrid peer-to-peer network that combines apeer-to-peer network model with a web-service to communicate to anotherpeer for monitoring and/or forwarding. Still other example embodimentsincorporate a hybrid peer-to-peer network that combines a peer-to-peernetwork model with a program (i.e., software) to communicate to anotherpeer for monitoring or and/or forwarding through a computer or server(e.g., chat server, relay server, Internet Relay Chat (IRC), etc.).

Hybrid peer-to-peer network architectures (schemes) generally may beimplemented to enable the communication between the monitored mobilecommunication device and the monitoring mobile communication device, asshown by example in FIG. 20. In the example architecture shown in FIG.20, the various lines indicate a link and communication channel betweentwo (or more) nodes which was coordinated by the server(s).

In an example operational method, upon an established link betweenmonitored and monitoring communication device(s), a “room” can beestablished on the server to facilitate a communication channel betweenthem. A “room,” for instance, may be implemented by a suitableprogramming tool such as a web application programming tool, aparticular nonlimiting example of which being Sockt.IO, a commerciallyavailable programming tool that can be used for example methods.Socket.IO is a JavaScript library for real-time web applications. Itenables real time, unidirectional or bi-directional communicationbetween web clients and servers.

In some example methods, the monitored device and the monitoring devicecommunicate with each other directly using the server to facilitate suchcommunication. Depending on the programming tool utilized for enablingthe communication, the term “room” may be used to describe it.

In some example methods, the monitored device sends one or more alertsand associated electronic messages (e.g., text messages, images, videos,audio, etc.) to the “room,” and a flag is raised (or other indicator ornotification is provided) to signal to the monitoring communicationdevice that there are alerts and electronic messages for it to download.Once the download is completed, the flag can be lowered (or otherindicator can be removed) to indicate that there are no more alerts andelectronic messages in the “room”. In example embodiments, dataassociated with a “room” (transmitted/received through a “room”) can betemporary; that is, not intended for long term storage or archiving.

An example hybrid peer-to-peer network scheme can be used as a mechanismto create a channel between two or more nodes (e.g., mobilecommunication devices), where only those nodes (e.g., grouped togetherfrom a pre-established link, have the credentials (or are otherwiseauthorized) to access the “room”. It can also be used to keep track of agroup of connected users. In some example embodiments, the “room” can bejoined or left at any time by the monitored and monitoring mobilecommunication devices (nodes). No flag is necessary to indicate thatstatus of data in this example embodiment.

Some example methods employing a hybrid peer-to-peer networkimplementation can incorporate a server (e.g., index server) tofacilitate electronic communications from the monitored mobilecommunication device to the monitoring mobile communication devicewithout permanently storing the electronic communication on the server.More specifically, files can be shared between nodes (e.g. child/parentmobile communication devices) as opposed to sharing through the means ofa centralized server. In such embodiments a connection between two ormore peers (e.g., socket, channel, session) can be established and datacommunications can flow in a unidirectional fashion (one-way) from onepeer to another (i.e., data flows only from sender to receiver orreceivers) or bi-directionally (i.e., where each peer can transmit andreceive data communications (e.g., FIG. 20).

Such example architectures can be configured such that electroniccommunications (e.g. data) are stored for a significant duration only onthe monitored mobile communication device and/or the monitoring mobilecommunication device, and merely flow through the server facilitatingthe connection and communications. However, it will be appreciated thatthe server may temporarily store the electronic communication (e.g.data) as it flows from the monitored mobile communication device to themonitoring mobile communication device.

Another example application of a hybrid peer-to-peer networkimplementation can be similar to the implementation disclosed above,except with the server replaced by a web service. By way of background,the term “web service” typically refers to a service offered by anelectronic device to another electronic device to enable communicationsvia the Internet or more specifically the World Wide Web. Other webtechniques, such as but not limited to Comet, can also be used tofacilitate or enable communication between nodes in a peer-to-peer orhybrid peer-to-peer network architecture over the World Wide Web.

In some example implementations of hybrid peer to peer networkarchitectures such as those described above the server may be consideredto act as (or analogous to) an “indexing server” containing theinformation about the location of each of the nodes in the network. Theserver can also act as a link between nodes to create a communicationchannel between two or more nodes, as shown in FIG. 20.

Some example methods for implementing a hybrid peer-to-peer networkenvironment employ asynchronous service to service communication used inserverless and micro-services architectures. One example technique ortechnology that can be used that incorporates this type of approach issometimes referred to as “Publish/Subscribe Messaging” or Pub/SubMessaging for short. With this example approach, the Publishers andSubscribers are a group of connections that can connect to each throughsome mechanism. The concept of a “room” as disclosed above is one suchtechnique. In example methods, the Publishers and Subscribers areanalogous to monitoring communication devices (e.g. parent/parents) andmonitored communication devices (e.g. child/children).

Example methods for capturing electronic messages (e.g. text messages,images, videos, audio, electronic messages, etc.) on a mobilecommunication device being monitored will now be descried. One examplemethod for capturing electronic messages is to read the input/outputbuffers that reside on the mobile communication device where data isreceived from or transmitted to another mobile communication device,computer, server, the Cloud, etc. These buffers are usually reserved forthis purpose and in some cases can be built into the hardware orfirmware of the device.

Another example method is to read the memory locations where apps orapplications store the input/output data being received from ortransmitted to another mobile communication device.

Another example method is to read the screen, perform a screen captureor capture/record a video stream occurring on the screen or other outputinterface of a mobile communication device being monitored. Thisrequires capturing (reading) the screen periodically or at particulartime intervals. In order to more diligently scan for targeted content,the screen capture or video stream capture operation can occurfrequently in order to ensure the vast majority of the content beingdisplayed on the screen, either inbound, outbound or previously storedis read and then subsequently compared against targeted or trainedtargeted content. One example approach is to attempt to capture/read thedevice screen (display) at least once per second. This will almostensure that no content that would qualify as targeted content will bemissed. The reason is that, for example, an average person usually takesat least one second to view an image, text message, etc. once it isdisplayed on the screen.

In some example embodiments, the monitoring of mobile communicationdevices for targeted content (e.g. images, MIMS, SMS, text messages,videos, audio files, electronic messages, etc.) can include analyzingcommunications for determining a match or a comparable match to suchtargeted content. Such analysis can employ, for instance, algorithms anddata structures/databases for determining the match or comparable match.

Some example methods can use artificial intelligence methods to performthis analysis. Generally, processor-based algorithms and techniques suchas machine learning (ML), neural networks, deep learning networks,convolutional neural networks (CNN), computer vision, machine vision,etc. are considered subsets of Artificial Intelligence. As used herein,Artificial Intelligence (AI) generally refers to the simulation of humanintelligence processes by machines (i.e., processor-based hardwareimplementing one or more software/firmware-based algorithms). Examplemachines include processor-based systems (e.g., computer systems)including but not limited to mobile communication devices. Specificapplications of AI include but are not limited to expert systems,natural language processing (NLP), speech recognition and machinevision.

More recent AI methods such as machine learning can predict or makedecisions based on sample data sometimes known as training data, whichin many cases creates a particular model that is used for this purpose.In some cases, certain models can increase in effectiveness or “learn”as they execute on future instances of data (e.g. images) provided(input) to it. One nonlimiting example machine learning architecturethat has been shown to be especially useful for image analysis (but canbe used for other analysis as well) is a Convolutional Neural Network(CNN).

In some example embodiments, one or more AI techniques, such as machinelearning, neural networks, deep learning networks, convolutional neuralnetworks, computer vision, etc., can employ algorithms, data structures,trained models from data sets, etc. to determine or help determine whatwould be targeted or potentially targeted content for determining adirect match or comparable match of an electronic message (i.e., image,video, audio file, text message, text message embedded into an image orvideo, etc.). To obtain an exact match across two images, for instance,is highly unlikely due to the nature of the data that comprises an imageor video frame (i.e., every pixel in the image would need to match).However, the ability to compare different images to targeted contentthat is of the same theme (i.e., akin to) but are not exact matches hasvalue in applications including example analysis methods herein.

As an example of possible targeted content, a selfie of a shirtless manmay need to be compared to images of other shirtless men trained into anAI model. The shirtless men in the trained AI model are not exactmatches to the image being compared against the AI model and therefore,the AI algorithm (AI model) can determine if there is a “comparablematch” (i.e, would be akin to). Machine Learning, neural networks, deeplearning networks, CNNs, etc. are nonlimiting examples of AItechnologies that may be used in computing devices, including but notlimited to mobile software apps, to provide enhanced clarification fordetermining a “comparable match”.

Some example AI algorithms and models for comparing different types ofcontent (e.g., image content, video content, audio content, textcontent) can optionally implement a rating or scoring method foridentifying similarities to targeted content. For example, an AIalgorithm (such as but not limited to a CNN) can return a rating levelof an image whereby the rating scale is 0.001 to 0.100 and a rating over0.050 is considered a threshold for a strong enough match or comparablematch. If the threshold is met, the computing device (e.g., themonitored user's mobile communication device, or child device can thengenerate an alert to be sent to the monitoring mobile communicationdevice (e.g., parent device).

Some example embodiments provide an AI algorithm with a trained model(e.g., trained with targeted content) for determining if the contentreceived, sent or displayed on the screen (e.g., screen capture, videostream capture) on a monitored user's mobile communication device is amatch or comparable match to targeted content or trained targetedcontent. In some example embodiments, this AI algorithm with a trainedmodel can reside (fully or partially) on the monitored user's mobilecommunication device whereby it is implemented as a mobile app or part(integrated into) a mobile app, is provided as an operating systemservice, is provided as a hardware/firmware system service, is acallable library function, an application component or any othersoftware manifestation on a mobile communication device known in theart. Some example embodiments use a CNN algorithm and trained model toanalyze for and determine if there is a match or comparable match tocontent received, sent or displayed on the screen (e.g. screen capture,video stream capture) on the monitored user's mobile communicationdevice.

For example, FIG. 21 shows an example process that can be performed bythe monitored user's mobile communication device (for instance) fordetermining whether incoming or outgoing communications on the monitoreduser's communication device matches or closely matches (i.e., is acomparable match to) targeted content, using an AI model trained withtargeted content. The monitored user's communication device captures,e.g., reads, incoming or outgoing communications and/or performs ascreen capture (captures what is currently on the screen, or a portionthereof), and then provides (e.g., inputs) the communication to an AIalgorithm with a trained model for identifying predetermined targetedcontent (e.g., a CNN algorithm and trained model).

The AI Algorithm returns a rating (e.g., on a scale of 1-100). If therating exceeds a predetermined threshold or a user selected threshold(e.g., 50), then a match or comparable match is determined, and thecommunication (or portions of communications that are determined to be acomparable match) is sent/transmitted to the monitoring user'scommunication device. Further, and as explained in more detail below,the communication (e.g. image, video) may be obfuscated (e.g., blurred)depending on the specific application.

CNNs are a useful type of AI technology that can be employed to identifycomparable matches within a class or category of images (e.g. detectingnudity in an image). Due to the nature of identifying this type oftargeted content within an image or video for monitoring purposes,utilizing an algorithm to find an exact match between two images wouldtypically not be practical. For instance, finding an exact match ofcorresponding pixels across two images would typically have little valuein an image monitoring application, since it is highly unlikely that animage generated by a monitored device will have a direct match to animage or images being used as targeted content. Similarly, the same istrue for a video and audio as it relates to a match or comparable match.

However, it can be useful to determine both exact matches and comparablematches for text message monitoring or monitoring of text messages thatare part of (incorporated into) an image. Text messages (i.e., words,acronyms) inherently lend themselves to exact matches. For example, twosets of letters that make up the same word that are in two differentfonts, have the same meaning.

In some example embodiments the AI algorithm with a trained model(trained with targeted content) on the monitored user's mobilecommunication device is incorporated (integrated) into a mobile appwhich transmits and receives data in a peer-to-peer architecture. Inother example embodiments, the AI algorithm with a trained model(trained with targeted content) on the monitored user's mobilecommunication device is incorporated (integrated) into a mobile appwhich transmits and receives data in a hybrid peer-to-peer architecture.

For instance, an implementation of a specific algorithm and datastructure/database that employs a CNN, deep learning, or other AI methodcan reside on the monitored user's mobile communication device, and canbe incorporated (integrated) into a mobile app which transmits andreceives data from the monitored user's mobile communication device toand from the monitoring user's mobile communication device in a hybridpeer-to-peer architecture. Some example implementations need nottransfer the data (e.g., image, video frame, electronic communication)to a server (e.g., in the hybrid peer-to-peer architecture) to performthe “comparable match” analysis by the AI model, but instead may utilizea server (e.g. index server, chat server, Internet Relay Server, etc.)merely for the purpose of facilitating peer-to-peer communicationbetween the monitored mobile communication device and the monitoringmobile communication device (in the hybrid peer-to-peer networkarchitecture). This can be implemented as unidirectional orbidirectional communication between the monitored mobile communicationdevice and the monitoring mobile communication device.

In other example embodiments, the example AI algorithm with a trainedmodel (trained with targeted content) can reside in the Cloud or server.In such embodiments, the monitored user's mobile communication devicecan transfer the incoming or outgoing communications or what isdisplayed on the monitored user's mobile device screen (e.g., screencapture, video stream capture) to the Cloud or server, where an AIalgorithm and trained model can be executed to determine if there is amatch or comparable match to the trained targeted content.

As provided herein, “Cloud” or “the Cloud” refers to the computerscience definition of the same for which one explanation is on-demandavailability of computer system resources, especially data storage andcomputing power, without direct active management by the user. Cloudcomputing typically shares computer resources to achieve uniformityamong shared resource and economies of scale (e.g. scale-up). Examplesof service providers of Cloud services today include but are not limitedto Google Cloud Platform, AWS (Amazon Web Services), Microsoft Azure andIBM Cloud. Examples of specific types of Cloud computing modelscommercially available today are IaaS (Infrastructure as a Service),Platform as a Service (Paas) and Software as a Service (SaaS). Allembodiments of this invention may also be implemented in a Hybrid Cloudarchitecture (scheme) which is essentially a way to connectinfrastructure and applications between cloud-based resources andexisting resources that are not located in the Cloud in addition tooptionally connecting to other types of Cloud resources (e.g.Multi-Cloud).

In other example embodiments, the implementation of the analysis (e.g.as performed using CNNs, deep learning, or other AI methods) todetermine a direct or comparable match can be processed in the Cloud(i.e. virtual servers). This example implementation can includetransferring data to the cloud to perform the “comparable match”processing. Other embodiments can include a similar implementation asCloud embodiment, except that the data to be processed can be sent to aserver (e.g. client/server architecture) to perform the processing todetermine a match or comparable match.

In still other example embodiments, a user or other party can select alevel of sensitivity for which images, videos, audio, text messages orother electronic communications are deemed to be (i.e. flagged as)inappropriate, NSFW (Not Safe For Work), illegal, offensive, etc. As anonlimiting example, the AI algorithm/model can be used to analyze animage for inappropriateness (e.g. nudity) and returns a rating of theimage as a value between 1 and 100 whereby 100 is the most inappropriatecontent (e.g., pornographic images). If the “factory default rating” orthreshold that needs to be exceeded in order to generate an alert to themonitoring mobile communication device is 50, for instance, this featurecan enable the end user (e.g., parent) to modify the default setting toa different level that would not be as sensitive (e.g. 60) or would bemore sensitive (e.g., 40). This could be implemented as a sliding scaleof sensitivity options or be a preset number of sensitivity levels(e.g., high, medium and low).

In other example embodiments, the inclusion of an age appropriatenessmonitoring function can be provided, which can be implemented using AI.For example, image recognition, audio recognition, video recognition,text message recognition, video recognition, etc., functionality can beused to decipher the age of the user being monitored (e.g., a child) inorder to alert a parent if the child is entering a false age (forthemselves) in order to, for instance, qualify with the terms on awebsite that is screening the age of their users prior to allowingaccess to their web content. A nonlimiting example is that thisfunctionality used in conjunction with or independently for websitesthat requires an end user to enter in their birthdate as a mechanism tocomply with the Child Online Privacy Protection Act (COPPA) law.

In other example embodiments, the implementation of either a specificalgorithm, a data set (e.g. training data for Artificial Intelligencemodels/algorithms) or both can reside in a Blockchain technologyimplementation (Blockchain architecture) on the Internet, as explainedin more detail below.

FIG. 22 is a flowchart illustrating an example process for providingcopies of obfuscated (e.g., blurred or distorted) image or videocommunications (e.g., electronic messages) from a mobile communicationdevice being monitored (which contains the original non-obfuscatedimage) to one or more mobile communication devices authenticated orregistered to monitor such communications in accordance with someembodiments. The electronic message is captured, e.g., read, and/or thescreen of the monitored mobile communication device is captured. It isdetermined whether the communication is a match or a comparable match totargeted content. If so, all or a portion of the communication isobfuscated. The obfuscated communication is then sent/transmitted to themonitoring mobile communication device.

The following are non-limiting examples of techniques that can beapplied in example embodiments to images or videos transmitted from themonitored mobile communication device to the monitoring mobilecommunication device for the purpose of obfuscating or blurring imagesor videos prior to them being transmitted to the monitoring mobilecommunication device: Image blurring, image obfuscation, imagepixilation (pixelating an image), pixelization, pixel interpolation,texture filtering, image resolution reduction, etc. Obfuscationtechniques for an audio message can include, for instance, audiblydistorting the audio file so that is less intelligible orunintelligible.

One example application for obfuscating images is to reduce or eliminatethe potential of an adult recipient (monitoring user) being transmittedcontent such as a pornographic image of a minor (i.e., under-agedperson) on their mobile communication device (e.g., smartphone, cellphone, tablet computer, etc.) which may be illegal. For instance, if theimage being transferred to the monitoring mobile communication device isthat of an under-aged person, then having content that resides on (or isstored on) their mobile communication device (e.g. smartphone, cellphone) would be a violation of the law in some countries. Theimplementation of a method for image blurring or image obfuscation orother image processing techniques prior to the transferring(transmitting) of images, videos, audio or other electronic messages toa monitoring mobile communication device (e.g. smartphone, cell phone)will help safeguard against inadvertently violating (breaking) the law.

One example for image blurring or image obfuscation that may beimplemented by example methods is via a technique referred to asGaussian blur, which applies a Gaussian mathematical function orfunctions to an image to obfuscate or blur it. Other variations of thistechnique include Gaussian smoothing, Gaussian filtering, and Gaussianimage processing. Other example techniques for blurring or obfuscatingan image that may be implemented by this invention may use low passfiltering to accomplish a blurring effect. Further example techniquesfor blurring or obfuscating an image that may be implemented includingusing a Convolution Matrix filter which makes use of matrices or masksto accomplish the blurring effect.

Still other example techniques for blurring or obfuscating an image thatmay be implemented in example methods include reducing or lowering theresolution (quality) of the image. An example of this technique would becompressing a JPEG image at various levels or percentages to create ablurred effect.

The following is a non-limiting list of terms and techniques for imagefiltering that may be used to implement (apply) a blurring effect orobfuscate an image: Kernels, blur effect, image convolution, imagematrix processing, compression, etc. However, this list should not beconsidered the complete list. It should also be noted that many or allof the same techniques and tools for obfuscating or blurring an imagecan be used to either enhance image quality or reverse a techniqueapplied to an image for obfuscating or blurring it. An example for theuse of this would be where the face portion of a person in an image wasobfuscated or blurred to obtain some level of anonymity. In the case of“Sexting” or sending an inappropriate (e.g., nude, inappropriategesture, etc.) picture to another person, it is possible that the faceportion of the image would be the only section of the image that wasobfuscated or blurred. It may be useful to reverse the obfuscation orblurring of the face to facilitate determining the identity of thesender in certain situations.

Some example embodiments can provide blurring or obfuscating variousportions of an image or video at different filter levels (e.g., levelsof blurriness). An example application of this is where the face of anude person in an image is obfuscated to a greater extent than the bodyor vice versa. There are situations where it may be necessary toobfuscate or blur the face portion of an image slightly and obfuscatethe body portion of the image more than the face portion. In someembodiments, portions of an image can be obfuscated of blurred atdifferent quality levels. For some example monitoring applications, itis feasible that when inappropriate targeted content is identified thatcontains nudity, the face portion of the image can be obfuscated orblurred to a lesser extent than the body portion of the image or viceversa. In this regard, algorithms and AI software tools may be employedif needed that can somewhat or completely reverse the blurring effect ofan image whereby the identity of the subject may be determined orrecognizable.

Some example embodiments enable the obfuscation or blurring of an image(or video) in two or more different segments of the image and at two ormore different filtering (quality) levels. For example, in the case ofthe transmission of a nude image (e.g., “Sexting”) the face portion ofan image of a person may be obfuscated at a higher quality level thanthat of the nude body. For example, this functionality may be necessaryto provide legal protection against child pornography by obfuscating orblurring the body (i.e., “neck down”) portion of the image to a lowquality level that is not deemed pornography while obfuscating orblurring the face (i.e., “neck up”) portion at a higher quality level sothat the identity of the person is distinguishable or recognizable tothe person receiving the image.

Additionally, in some embodiments an end user may be able, e.g., byinteracting with the software (App), to set the level or levels offiltering for the obfuscation or blurriness. For instance, this mayinclude being able to set the filtering levels of various portions of animage independently. As a nonlimiting example, this can enable theobfuscation of the face portion of an image to be set at a differentquality level than the quality level of the body with the ability toadjust each quality level independently.

This could also be implemented with three or more independentlyadjustable quality levels for three or more portions of an image. Ingeneral, the setting of levels of quality of obfuscation or blurrinesscan be implemented at up to nearly an infinite number of levels and canbe set for one, two or more sections of an image independently, e.g., bythe software (e.g. App), the end user or a combination of both.

Some example embodiments can additionally provide for the detection ofadversarial content, adversarial explicit content, adversarialpromotional explicit content or malicious content by a mobilecommunication device, whereby an alert can be generated with an optionalcopy of the content in question and an optional description of thecontent. Example methods can apply one or more software techniques(e.g., AI, machine learning, neural networks, deep learning networks,CNNs, Gaussian filtering, Gaussian image processing, etc.) to determineif an image or video was obfuscated (blurred) to a filtering level or bysome other means that would enable it to pass (not create an alarm) aNot Safe For Work (NSFW) safety check/scan (or any other content safetycheck/scan). It should be noted that NSFW is used herein merely as anexample application, and it is not intended to be a limit of the typesof applications that it can be used/implemented for.

For instance, there are ways of transferring information in a somewhatstealth form for it to not be flagged by automated methods for detectingsuch content that may or may not be commercially available and used on awide scale. As one example, an adversary may try to use the adversarialpromotional explicit images to evade inappropriate image detectors forpromoting illicit products (e.g., sexual products, gamble sites, illicitonline pharmacies, etc.). The adversary may obfuscate the image, usingvarious distortion or filtering techniques (such as noise, blur andocclusion) at a distortion (or filtering) level that that suchadversarial promotional explicit images can still be correctlyrecognized by humans. Example methods can be used to detect such contentin communications.

Another non-limiting example is detecting content in military, nationaldefense or law enforcement applications, whereby content may beobfuscated in a way that enables text, audio, image or video content tonot be detected or easily detected by services, software, humaninspection, surveillances or any other means for inspecting for suchcontent that adversaries would utilize for their purpose or cause.

Another non-limiting application example is detecting content producedfor attempted evasion by an adversary from online filtering tools suchas the Google SafeSearch tool, which attempts to filter out pornographicand other potentially offensive content during a web search. Anothernon-limiting example of types of content that may be scanned for byexample methods is adversarial promotional pornographic images (APPIs).

FIG. 23 is a flowchart illustrating an example process according toadditional embodiments for determining if an image or video that resideson a monitored user's mobile communication device has characters, wordsor sentences embedded into the image or video. An example process usesArtificial Intelligence (AI) algorithms/software/technology. In FIG. 23,dashed lines denote that the functionality grouped together may beprovided by a single function, subroutine, library call, web service,etc.

The communication is read or captured, and analyzed using AI, examplesof which are provided above. If the AI system determines that the imageor video has characters, words or sentences, then Optical CharacterRecognition algorithms/software/technology (OCR) can be utilized toextract the characters, words or sentences from the image, whereby theycan be stored in a format that is typical for characters such as ASCII.In some embodiments, an AI algorithm/system can include integrated OCRso that, for instance, an image is sent to it and it returns charactersin the image if they exist.

After the characters, words or sentences are converted to a computerformat that allows for analysis, they can be compared to predeterminedtargeted content (e.g. library or words, acronyms, sentences) and/oranalyzed by an AI system (e.g. Natural Language Processing (NLP)algorithms and model trained with a predetermined data set) withpreviously trained targeted content for a match or comparable match. Ifa match or comparable match is found (determined), then the image orvideo frame may be optionally obfuscated depending on the specificapplication and then sent (transmitted) to the monitoring user's mobilecommunication device. According to some embodiments, an alert (i.e.notification) may be also be sent to the monitoring user's communicationdevice to alert the monitoring user (e.g. parent) of the incoming imageor video frame(s).

Other example embodiments decipher one or more of text, characters,written words (e.g. cursive, script), numbers, special characters,sentences, messages, etc. that are imbedded or integrated into an image(i.e., part of an image) or video (i.e., part of one or more frames ofvideo). More specifically, the text, characters, written words, numbers,special characters, sentences, messages, etc. in the image are notstored, for example, as individual characters like ASCII characters arestored in the memory of a computer, server, mobile communication device,etc. The text, characters, written words, numbers, special characters,sentences, messages, etc. are incorporated inherently into the imagefile itself.

For example, FIG. 24 shows an image/picture of a balloon that has thewords “Happy Birthday” written on it (e.g., stylized). Example methodscan use a combination of Artificial Intelligence (AI) and OpticalCharacter Recognition (OCR) as shown in FIG. 23 to decipher andtranslate this information into characters or character strings that canbe used to perform a direct match to targeted content or a comparablematch to targeted content (or trained targeted content) that is storedin a format like ASCII.

Another example is processing an image from the user of a social mediaapp that allows the user to take a picture and overlay words on or overthe picture (similar to a caption). An example method uses AI technology(e.g., a CNN) and OCR technology in conjunction or cooperatively witheach other. For instance, the AI technology can first be used todetermine or identify that text, characters, written words, numbers,special characters, sentences, messages, etc. are in the image(picture). OCR technology can then be used to convert the text,characters, written words, numbers, special characters, sentences,messages, etc. in the image into characters (e.g. ASCII characters),words, sentences or messages, which are in turn used for comparing,matching or determining a comparable match to targeted content (ortrained targeted content). The comparison may use the same or anothertype of AI technology for making this determination or for merelycomparing each character in the translated character string to eachcharacter in the targeted content character string to identify a match.

One example application of such methods allows parents to be alertedwhen communications related to bullying (cyber-bullying) have occurredon a monitored mobile communication device through the use of a socialmedia app that allows text, characters, written words, numbers, specialcharacters, sentences, messages, etc. to be overlaid over an image orvideo prior to transmitting it. These social media apps can also bereferred to as “image messaging media apps” and “video messaging mediaapps”. Current examples of these types of apps include Snapchat andTikTok.

It is important to note that the order of the processing and thecombination of the processes utilized for the AI and OCR in thisembodiment can vary. For example, in some applications, the image may beprocessed with AI technology to first determine that there is in facttext, characters, written words, numbers, special characters, sentences,messages, etc. in the image. Provided the AI determines that text,characters, written words, numbers, special characters, sentences,messages, etc. do exist within the image, then OCR can be used as asecondary function to convert into characters that can be stored in acomputer (e.g. ASCII format). After that, AI may be used to determine ifthe characters (e.g. words, acronyms, sentences, messages) have a match,comparable match or are sufficiently close match to targeted contentwhich will create an action (e.g. an alert sent to a parent). Thisinvolves an additional analysis process. After that, another additionalAI analysis process may be incorporated, which will determine if theimage has targeted content not associated with the text, characters,etc. (e.g., NSFW, inappropriate content, an adolescent smoking/vaping inthe image, a firearm or weapon in the image, etc.). Each of the aboveembodiments for identifying a direct or comparable match can be used fordetermining a match or comparable match for content which includes butis not limited to text, image, video, audio, combinations oftext/image/video/audio or other types of electronic messages.

Some example embodiments can employ blockchain methods. By way ofbackground, a blockchain, originally block chain, is a growing list ofrecords, called blocks, which are linked using cryptography. Each blockcontains a cryptographic hash of the previous block, a timestamp, andtransaction data. By design, a blockchain is resistant to modificationof the data. Blockchain applications are typically implemented using apeer-to-peer architecture or in a peer-to-peer network and are alsodecentralized network implementations for data flow, data storage,control, etc.

A hybrid peer-to-peer network implementation of blockchain has certainadvantages for example embodiments. There are both public and privateblockchain implementations, and although some example embodiments lendthemselves to a private blockchain implementation whereby the users whocan access it are authorized, registered or invited, other embodimentscan also be implemented in a public implementation. One advantage forincorporating certain embodiments via a public blockchain implementationis that it provides greater accessibility to resources and distributionof data, machine learning models and information with less opportunityfor manipulation. Therefore, the implementation of both private andpublic blockchain for a single application of example embodiments isviable.

Some example embodiments incorporate blockchain architectures (i.e.,Blockchain technology) and schemes to store some or all of theelectronic communications, the status of matches or comparable matches(e.g., deep neural network model techniques for determining a comparablematch) to targeted content and alerts. The data set(s) or modelsutilized with AI technologies or other comparison techniques likelibraries, arrays or knowledge bases can also be stored usingblockchain. This enables a decentralized technique that has manyadvantages including enhanced data security, improved traceability,increased speed and efficiency.

Some example embodiments incorporate blockchain architectures (i.e.Blockchain technology) and schemes for implementing a system or processfor granting authorization (“permissioning”) between two or more mobilecommunication devices. In some embodiments, this can include authorizingone or more monitoring mobile communication devices to receiveelectronic communications from a specific monitored mobile communicationdevice.

Some example embodiments incorporate Blockchain architectures (i.e.,Blockchain technology) and schemes to store information pertaining tothe identity (e.g. Internet address, location, etc.) of the partiessending (transmitting) electronic communications to the user of themonitored mobile communication device. Some advantages to this approachinclude improved traceability and security. Other information could alsobe stored in a Blockchain as it relates to useful information such as,but not limited to, data flow information, network information, Internetinformation (e.g. address information, URL, IP address, protocolinformation, routing information), control information, transactioninformation, etc.

Some embodiments provide the capability to analyze for targeted content(e.g., inappropriate content, offensive content, or any content ofinterest) through the interpreting of organized sets of data (e.g.,training data) of interrelated blockchains.

There are other available types of technologies and techniques that aresimilar to blockchain in that they utilize blocks of information withdata and pointers to associated blocks (e.g., hash) which aredistributed across nodes (e.g. computers) connected by the Internet in adistributed architecture. One open source technology or technique iscalled Hyperledger, which can be implemented in example embodiments inthe same ways as described above for blockchain. Similarly, thefollowing non-limiting list of technologies or techniques can also beused for the example embodiments disclosed above: Corda, Quorum,Euthereum, Holochain, Ripple, Stellar, etc.

Additional features of example embodiments for monitoring mobilecommunication devices for targeted content or trained targeted content(e.g., images, MMS, SMS, text messages, videos, audio clips, electronicmessages) will now be explained, including example features for readingor receiving electronic communications on a monitored user's mobilecommunication device, with reference to FIG. 25. FIG. 25 is a flowchartillustrating an example method for capturing, analyzing, andtransmitting an image or recording a video stream being presented, e.g.,displayed, on an output interface, e.g., a screen, of a mobilecommunication device.

A mobile communication device, e.g., the monitored (child) mobilecommunication device, captures presented images or video from acommunication by performing a screenshot or recording what is displayedon the mobile communication device screen. Example capturing can beimplemented on a regular basis (e.g., based on a sample rate, imagecapture rate, etc.) and/or based on predefined or user-defined criteria.Example methods can capture images or video that are displayed on themobile communication device screen at a predefined time interval(synchronous scan rate), within a timeframe (asynchronous scanning),and/or based on specific criteria.

An example method can employ one or more screen capture techniques,technologies, customized or proprietary software or firmware, commercialor open source software, or any combination, executable by a processorto read the screen (e.g., take a screenshot), capture the screen orcapture a continuous video stream of the monitored user's mobilecommunication device. Programmatically, capturing the mobile devicescreen can be implemented by, for instance, accessing system levelfunctions of the operating system, standard operating system functionsor calls, through software or apps running on a mobile device, APIfunctions, by accessing and reading video memory (e.g. RAM, VRAM, DRAM,NVRAM), by accessing “working memory” where images or videos aretemporarily stored, high-speed memory or through any other manifestationfor capturing the contents on the screen of a mobile communicationdevice known by someone with skill in the art. Memory can also includevolatile or non-volatile types of memory.

Similarly, continuous video streams can be captured from the mobilecommunication device screen whereby individual or groups of video frames(images) can be used for analysis and processing. Software used for thisfunction that is executable by a processor can be embodied inproprietary code, software developed by another party (e.g. open sourcesoftware, commercially available software), or a combination.

Video memory refers to the area in memory on a processor such as amobile device or computer where an image resides prior to beingdisplayed on the device screen for viewing. Video memory may reside, forinstance, in the RAM (Random-Access Memory), VRAM (Video RAM), DRAM(Dynamic RAM), NVRAM (Non-Volatile RAM) or other similar types of memoryused to store “working data” of a mobile communication device. Inaddition, it is also possible to intercept or capture an image or videoin digital format as it is transferred from video memory (e.g., VRAM)and prior to it being displayed on the mobile device screen. Examplecapturing, analysis (e.g., AI algorithmic), content processing (such asobfuscation), transmission, and other processes providing one or morefunctions of example embodiments can, in certain cases, be executed inparallel or as a multi-threaded application.

Referring again to FIG. 25, once an image or video stream is captured orrecorded, the image or individual video frame can be presented to AIsoftware (e.g., a CNN trained model) for analysis. Predicated on a matchor comparable match (i.e., a closely related match) to targeted contentor trained targeted content (e.g., via the CNN), the image or videoframe can be transmitted to the monitoring user's communication device.The image or video frame may be obfuscated prior to transmission basedon the application, as explained elsewhere herein.

According to some example embodiments, an alert (e.g., a notification)may be also be sent to the monitoring user's communication device tonotify the monitoring user (e.g., parent) of the incoming image or videoframe.

Another embodiment of this invention allows the monitoring user's mobilecommunication device to receive “present-day checks” of what types ofcontent the monitored user is sending, receiving or viewing on his/hermobile communication device. The present-day check functionalityprovides a monitoring user (e.g., parent) a way to more actively monitormobile communications occurring on the monitored user's mobile device(e.g., child). Note that “present-day” is not limited to a daily check(though this is possible), but can instead refer to checks within anytime frame.

The present-day check examples and descriptions provided herein canadditionally or alternatively enable the monitoring mobile communicationdevice to receive incoming and outgoing electronic messages (e.g.,screenshots, images) from the monitored mobile communication device thatare not considered a match or comparable match to targeted content(e.g., a selfie picture not deemed to be inappropriate). For example,the present-day check feature can enable the monitoring mobilecommunication device (e.g., parent) to monitor what is happening on themonitored user's mobile communication device by receiving an ordinaryimage, screenshot, text message or other electronic communication,periodically and/or based on predefined criteria and/or on-demand.

As an illustrative example, it may be beneficial to monitor what the enduser (e.g., a child) is looking at on the screen of his/her mobilecommunication device from time to time. Information transferred to amonitored mobile communication device but not viewed by an end user mayhave less value (i.e., may be less important) to a monitoring user(e.g., parent).

FIG. 26 shows an example of a “present-day check” method. A screendisplayed from a communication (e.g., a screenshot, video stream,inbound/outbound communication, etc.) on the monitored mobilecommunication device is captured or recorded, as provided above. Ascreenshot can be an image, such as but not limited to a “selfie”picture or merely a snapshot of what is happening on the monitoreduser's device screen at a particular moment in time (e.g., a snapshot ofa game being played).

A timing method may be utilized for providing a screen capture samplingrate in addition to the period in which the best quality present-daycheck screenshot is transmitted to the monitoring user's mobile device.For instance, by conducting a screen capture at a predefined setinterval (as a nonlimiting example, a screenshot taken once everysecond) or based on other criteria, this type of monitoring can beeffective for various applications. The more frequently the screen iscaptured and processed, the more effective and thorough the monitoringfunction may be for some applications.

To analyze the content, e.g., the captured image or video frame, acontent recognition algorithm/model can be implemented using AItechnology or any other type of image recognition software or techniquesthat rate images against targeted content, as provided above. Forexample, content can be rated by a Convolutional Neural Networkalgorithm/software (CNN) for determining that an image (screenshot) hasa comparable match.

Based on this determination, the analyzed content can further be ratedby the example AI algorithm (or an additional AI algorithm) from, say, 1to 100, where 100 is the most inappropriate. The specific rating (e.g.,1 to 100) can indicate how close of a match the content is to thepreviously defined targeted content. However, content rated lower thanthe rating level that indicates a good match (e.g., rating of 85 on ascale of 1 to 100) may still have meaningful value to a monitoring user,for example to provide a typical snapshot of what is happening in thelife of the monitored user (e.g., present-day check).

As a nonlimiting example scale: content rated >50 may be considered tobe inappropriate (e.g. NSFW—Not Safe for Work); content rated >85 may beconsidered to be highly inappropriate (e.g. Pornographic); content rated<20 may be considered to be appropriate (e.g. SFW—Safe for Work); andcontent rated between 10 and 20 may be considered likely to show a humanimage rather than a screenshot of something non-human (e.g. the icons ona user interface on a mobile device). Note that the above thresholds arefor purposes of explanation only and can vary significantly dependingon, for instance, the AI algorithm and training model used or thetolerability of the end user, etc. The scaling may vary as well (e.g.,rating scale could be between 0.001 and 0.100 instead of 1 and 100, bereversed (from 100 to 1), or vary in other ways).

In some example embodiments, if the content analysis determines at leasta comparable match (i.e., a closely related match) to targeted contentor trained targeted content (e.g., by employing a CNN trained model),then the image or video frame may be stored on the monitored user'scommunication device for a future transmission based on predefinedcriteria. The timing, criteria and/or protocol or transmission (e.g.,the number of electronic messages, images, screenshots, etc.) from themonitored mobile communication device to monitoring mobile communicationdevice can vary substantially in, for example, the exact number ofelectronic messages (e.g., images, screenshots) transferred, thecriteria for which electronic messages (e.g., images, screenshots) areselected, the specific timing (or randomness) of when the electronicmessages (e.g., images, screenshots) are captured or are to betransmitted, the cutoff point or threshold for the rating, a score orlevel that qualifies an electronic messages (e.g., images, screenshots)for transmission to the monitoring mobile communication device (e.g., arating from a CNN algorithm), etc.

In an example operation, the example application operating on themonitored user's mobile communication device may be configured totransmit only the highest rated image within a predetermined thresholdrange (e.g., a rating between 5 and 20 on a scale of 1 to 100), and at aset interval (e.g., transmit after every 6 hour time period). If thenext image or video frame analyzed by AI has a higher rated match (i.e.,a better match) or higher rated comparable match than the previous imageor video frame, then it can replace the current one stored. After apredefined timeframe or in response to an on-demand request by the userof the monitoring communication device, the image or video frame can betransmitted to the monitoring user's communication device (which may ormay not be obfuscated prior to transmission based on the application).

In another example embodiment, the monitored communication device maytransfer one or more random pictures at a set time interval (e.g. as anonlimiting example, every 6 hours). The device can determine whichscreen capture/image(s) to transfer to the monitoring communicationdevice by storing the highest rated picture that does not exceed aspecific threshold in order to insure the image was not NSFW content butis likely to have more meaningful content to the monitoring user thanany random screenshot.

One way that this can be accomplished is by using AI (e.g., CNN) thatwas previously trained for targeted image recognition and have multiplethresholds for what is to be considered a present-day check screenshotor image versus an NSFW screenshot or image. A dead band or gap may alsobe implemented between the rating thresholds. One example benefit forthis is that by staying under a particular threshold, for example: 20 ona scale of 1 to 100, this would allow content to be transferred thatwould not be deemed inappropriate or would be SFW (Safe For Work)content; for instance, with the example rating scale above the NSFWthreshold would be a rating greater than 50. The screenshots or imagesthat fall into the dead band range (or gap) can be ignored. Anothernon-limiting example would be for screenshots or images to betransferred without regard to a rating and be determined based on aspecific time period, a random time period or any other criteria.Additionally, in example implementations of this embodiment, thescreenshots or images may be optionally obfuscated (e.g., blurred). Thiscan provide, among other things, a level of anonymity.

FIG. 27 shows example screenshots of a user interface (i.e., means bywhich a human and a mobile communication device or computer interact) ofa monitoring user's mobile communication device. FIG. 27 shows twoexample views of transmitted images/screenshots, namely a thumbnailgallery view of images or electronic messages (left) and a single imageview (i.e., zoomed in view of a single thumbnail image) of an image orelectronic message (right). A burst thumbnail view can also beimplemented, e.g., showing a plurality (as a nonlimiting example, four)images within a thumbnail with a number displayed on it that reveals thetotal number of images in the burst. A burst is a series of images thatwere taken in a short period of time. One non-limiting example where aburst of images is taken would be a video being viewed on the monitoredcommunication device that has targeted content (e.g., pornographicvideo).

Additional example embodiments integrate example methods for reading orcapturing content on a mobile communication device and determining if amatch or a comparable match to targeted content exists with/into asurveillance or home security system. In example embodiments, any of theimage recognition, video recognition, text message recognition, audiorecognition and electronic communication recognition functionalitydisclosed above can be employed. The implementation can be standalone orintegrated into existing or to-be-developed home security orsurveillance systems, such as by providing suitable code (e.g.,software, firmware) executable by a processor in the home security orsurveillance systems for performing example methods.

One non-limiting example embodiment can provide additional functionalityfor alerting a user (e.g., home owner, business owner) of aninappropriate image, text message, video, audio recording, SMS, MIMS,electronic communication, etc. being transmitted into or out of theirhome or business via wireless, cellular, Internet, Bluetooth, or anyother communication technology or medium. An example implementation ofthis embodiment to work in conjunction with a home security system caninclude, for instance, a screen (similar to that on a monitored mobilecommunication device) for the software to capture the images beingtransmitted through the electronic communications network (e.g.,Internet). However, it is also possible to read these electronicmessages through other means (e.g., reading video memory, input/outputbuffers, etc.). Thus, while current security systems safeguard againstphysical intrusions, example methods incorporated into a security systemcan safeguard against “digital intrusions”.

Additional example embodiments can employ Artificial Intelligence,Machine Learning or other human cognitive function technologies oranalysis functions as described above to perform the same (or similar oranalogous) functions on content stored on a mobile communication devicebeing monitored (e.g., previously stored images, videos, audio, textmessages, electronic messages, etc.). This can provide, for instance, adevice scan feature. Some example methods can search archived or storeddata on the mobile communication device for targeted content. Examplemethods can additionally or alternatively be used to determine if otherapps installed on the mobile communication device being monitored areinappropriate or do not meet other standards that may be harmful orotherwise of interest to a monitoring party.

The results of the scan can be posted to a report that can betransmitted (e.g., emailed) to a recipient, such as a parent. After thedevice can is finished, results can give the user the list of detectedcontent, and may allow the parent to delete said content individually orin its entirety.

In some example embodiments, the Artificial Intelligence analysisprocess or other technologies that mimic human cognitive functions(e.g., Machine Learning algorithm/model, deep learning algorithm/model,etc.) may be incorporated to determine a variety of different targetedcontent. Such content need not be restricted to targeted content thatmay be, for example, of interest of parents when monitoring anadolescent's/child's mobile communication device. Some non-limitingexamples of additional targeted content include a person smoking/vapingin the image, a firearm or weapon in the image, a person wearing a maskto conceal identity, identifying specific behavior, identifying specificperson or persons, a person falling asleep while driving, a personhaving a seizure, a person showing signs of a specific illness ormedical condition, a person being in a place that they should not be,facial expressions or body language indicating depression, facialexpressions or body language indicating mental illness, facialexpressions or body language indicating suicide potential, facialexpressions or body language indicating anger, facial expressions orbody language indicating sadness, facial expressions or body languageindicating contempt, facial expressions or body language indicatingconfusion, facial expressions or body language indicating fear, facialexpressions or body language indicating surprise, facial expressions orbody language indicating excitement, etc.

Other example embodiments provide a computer appliance, illustrated inFIG. 28 as a “box,” for performing example monitoring methods asdisclosed herein. The computer appliance may be disposed between one ormore content providing devices, e.g., streaming devices, cable box,etc., and an output device, such as a television or other monitor ordisplay device. The computer appliance may be a standalone device or maybe integrated into a device used to select or forward content, such asbut not limited to an HDMI switch, a television or monitor front-end, anaudio/video receiver, a computer, a gaming console, etc.

The example computer appliance can contain suitable hardware andsoftware (e.g., circuitry and logic) for determining if a video streamor other input content stream that is input from outside devices, e.g.,cable box, streaming devices, etc. contains any targeted content (e.g.,images, videos, audio, or text deemed inappropriate for children underthe age of 18). If such targeted content is determined, then the contentthat matches or has a comparable match can be deleted from the video orother content output stream so that they do not appear on the outputdevice.

Determination of whether or not the content of the video or other inputcontent stream meets this level of inappropriateness can be performed byan Artificial Intelligence algorithm with a trained model from apredetermined data set (AI). For example, this may be driven by a CNNthat has been custom trained to recognize inappropriate images andwords. In some embodiments, the example computer appliance can takesingle frames that are captured from a live or real-time video input,and format them into a digital array that the CNN can read. This arraycan be input to the CNN or other AI model, where a rating or series ofratings is given for each category that is inappropriate. In anonlimiting example, ratings can be in the range of 0.001 to 0.100,where any rating over 0.050 is considered a positive match or comparablematch.

As shown in FIG. 28, the computer appliance can include multiple videoinputs and one or more video output (one shown in the figure). An inputselector may be provided for selection among content providing devices.The example computer appliance also has Wi-Fi (or other wirelesscommunication) capabilities in order to communicate information to amonitoring user's mobile devices or computers. Information that can betransmitted between the computer appliance and the mobile device may betransmitted done through, for instance, a peer-to-peer, hybridpeer-to-peer or hybrid peer-to-peer Cloud network that is secure andprevents any images from being stored permanently or archived online(e.g., centralized server). Images or video frames or other inputcontent that has been determined to have a match or comparable match totargeted content (i.e., flagged content) can be forwarded to themonitoring user's mobile device along with an alert via one of thenetwork schemes (e.g., as described herein), for instance to a mobileapp residing thereon in order to present (e.g., display or otherwiseindicate) the content that has been flagged. The alert may be deliveredvia a mobile app, a push notification or other means outside of the appitself.

In an example embodiment, the computer appliance (including plug-ins, ifused) contains the necessary circuitry and executable logic to convertthe video streams from input content into digital format, capture singleframes of said video at fixed intervals such as (but not limited to)once every half a second, pre-process those single frames for input tothe AI for analysis, determine the ratings the AI assigns to each frameand compare it to a predetermined threshold (or high & low thresholds)to see if it exceeds the threshold or is in between the high and lowthresholds. If a single image or video frame (or other analyzed content)exceeds the threshold requirements, it is transmitted, e.g., via Wi-Fiand one of the network schemes (described herein) to the monitoringuser's mobile device, and can be deleted (e.g., dropped) from the videostream.

An example AI model may reside on a special chip called an EPROM(Erasable Programmable Read-Only Memory), a type of memory that can beerased and reprogrammed for updated versions of the model (e.g.retrained models with new targeted content). The computer appliance may,but need not, further include specialized hardware as will beappreciated by an artisan that can be sourced to run the AI at animproved efficiency, e.g., TPU AMD GPU. The example computer appliancemay also be configured to communicate with external devices such as butnot limited to remote controls, or Apps or other device for allowing theobserver of the TV or external monitor to select which input videostream should be used.

Example embodiments of the near real-time electronic messaging (with anexample being text messaging) monitoring feature disclosed hereinprovide, among other things, a method for monitoring text messages orSMS occurring on one or more monitored mobile communications devices(with examples including, but not limited to, tablet PCs, laptops,gaming devices, music players, PDAs, mobile communication devices,smartphones, cell phones (e.g. children's cell phones)) by anotherparty's monitoring mobile communications device (e.g. parental cellphone).

Other embodiments provide a method for transmitting incoming and/oroutgoing video content or clips that match or reasonably match (closeproximity match) a predefined video content/clip stored in a library ordatabase on either the monitored or monitoring mobile communicationdevice. One example use of such embodiments is to enable a parent tomonitor video content for selected images, moving pictures or othervideo identifiers. This allows a parent to monitor incoming and outgoingvideo content occurring on their children's cell phone(s). Alerts canalso be sent to the monitoring party's cell phone indicating when amatch or close proximity match has occurred. Once the alert isrecognized by the parent, they can watch the actual video content thattriggered the alert from the convenience of their own cell phone ormobile communication device. In such embodiments, video content mayinclude the audio content accompanying the video whereby either thevideo content and/or the audio content (similar to the method in theprevious provision that monitors audio phone conversations), can bemonitored for generating alerts.

Still other embodiments provide a method for transmitting only theincoming and/or outgoing text messages to a third party cell phone thatare associated with one or more preselected contacts (contact names)and/or phone number(s) and/or other communicating identifier by themonitoring party (e.g. parent or guardian). This enables a parent tolimit the text messages being monitored based on certain contact(s)and/or phone number(s) that they preselect.

Other example embodiments provide a method for alerting the monitoringcell phone(s) when potentially inappropriate, illegal or targetedcontent is detected within a text message, SMS, MMS, audio message(words communicated through voice), audio/video message, video, images,or electronic message or during a phone (audio) conversation. Thisincludes words, acronyms, shorthand, codes, audio clips, voice excerpts,sounds, images, image snippets or other identifiers that are, or areknown to have meaning that may be considered inappropriate or illegal ordefined as targeted content. The application software running on themonitored device (e.g., cell phone) can have a library of words,acronyms, short hand, codes, images, image snippets, audio clips, voiceexcerpts, video and other identifiers stored in a database that may beillegal or considered inappropriate by a parent, guardian or society.When this type of inappropriate content is detected by the applicationsoftware running on one or more of the cell phones being monitored, analert is sent to the monitoring device (e.g., see FIG. 13) fornotification purposes.

One intended use for this embodiment is to notify the user of themonitoring device, in near real-time, that the user of the monitoreddevice may be engaged in or was engaged in inappropriate or illegalelectronic communications (e.g. text message, MMS, audio message (wordscommunicated through voice), audio/video message or electronic message).Updates to the database (library) of words, acronyms, short hand, codes,images, image snippets, audio clips, voice excerpts, video and otheridentifiers are (or can be) provided to the end users as part of aservice model for this technology and are uploaded to the database onthe monitored device (e.g., cell phone) periodically.

In example embodiments herein, the method in which these alerts areexecuted during run-time is in a peer to peer scheme (i.e. peer to peernetwork architecture). That is the library of inappropriate words,acronyms, short hand, codes, images, image snippets, audio clips, voiceexcerpts, video and other identifiers (“Watch Words”) and also taughtherein as targeted content is stored on the cell phone, smartphone ormobile communication device directly. In addition, the mobile appsoftware that evaluates the Watch Words or targeted content and receivesand displays alerts is stored on the Monitored Party's or MonitoringParty's cell phones (mobile communication devices) and not on any otherintermediary server.

Another embodiment is a method for monitoring electronic communicationscomprising the steps of:

-   -   Receiving or communicating a first electronic message from or to        a third party with a first mobile communications device;    -   using a first application stored on the first communications        device to monitor and store the first message;    -   using the first application to communicate a second message that        is a copy of the first message or includes information        describing at least some aspect of the first message to a second        application that is stored on a second mobile communications        device; and, using the second application to receive the second        message and to display the information.

Still another embodiment is a method for monitoring electroniccommunications on a mobile communication device by reading thecharacters, text strings, images, audio and videos generated by the user(“Input Elements”) and then, analyze and compare the Input Elements toor against predefined text strings, images, audio content, audio/videocontent and/or videos stored in a local data structure or database. Incontrast to SMS and MIMS communications which utilize a wirelesscommunications provider's network (cellular network) for transmittingand receiving communications and may use the mobile communicationdevice's (smartphone) operating system and communication buffers (i.e.SMS buffer or MIMS buffer) to facilitate the occurrence of a SMS messageor MIMS message (for example, an interrupt generated by the operatingsystem to tell the software/app that a new SMS message was received) andthe replication of such a SMS message or MIMS message (for example, thecopying of a SMS message from a communication buffer (SMS buffer) to adatabase). In contrast, this embodiment enables the monitoring ofcommunications to/from messaging applications (i.e. mobile apps thatincorporate SMS-like functionality, MMS-like functionality or both),also referred to as Mobile Messaging Apps, that currently utilize Wi-Fi,Bluetooth or other wireless technology to connect a smartphone or mobilecommunication device to the Internet to transmit and receive textmessages, images, audio content, audio/video content, electronicmessages and videos. Example mobile apps for this are, but not limitedto, WhatsApp, MessageMe, KiK, TextNow, WeChat.

Input Elements include user interactions on a mobile communicationdevice that include, but are not limited to, entering in characters on akeyboard, on-screen keyboard or voice recognition mechanism ordepressing icons on a mobile communication device that activate apicture/image to be taken and/or audio/video to be captured. In onecircumstance, the input/output buffers associated with the mobilecommunication device's operating system is not utilized for transmittingand receiving messages, images and videos, the application software isrequired to analyze the users input entered in on the mobilecommunication device to determine the communication being transmitted.For example, this could be a text message, image, audio clip,audio/video clip, electronic message or video being transmitted over theInternet via a Wi-Fi, Bluetooth or other wireless technology connection.After analysis is performed, the communication is compared to apredefined database or data structure of text strings, images, audioclips, video clips or target content (“Target Element Data”) stored onthe Monitored Party's or Monitoring Party's mobile communication device(FIG. 3). The comparison to the Target Element Data may also occur onthe monitoring data communication device in addition to the TargetElement Data being stored on the monitoring data communication device.Provided there is a match or a close proximity match, then thecommunication and/or associated alert (warning) is transmitted to themobile communication devices optionally authenticated or registered tomonitor such communications. Further, the software may obtaininformation about applications (mobile apps) running on the monitoredusers' mobile communication device from a monitored user's mobilecommunication device's operating system. This information may be used tofacilitate the parsing and analysis of text messages, images, audio andvideo based on previously determined communication protocols, datastructure definitions, formatting definitions, services, characterformatting and/or device input protocols or formats learned or knownabout the protocols, data structures, formats, techniques or servicesimplemented by each specific application (mobile app).

Again, it is preferred that the example application software is intendedto enable communications, most preferably during run-time, between twoor more mobile communication devices (e.g. cell phones, smartphones,iPads, iPods) in a peer to peer network architecture (scheme).

The summary and embodiments described herein are to be understood asbeing in every respect exemplary, but not restrictive, and the scope ofthe invention disclose herein is not to be determined strictly from thesummary and embodiments, but rather as interpreted according to the fullbreadth permitted by the patent laws. It is understood that theembodiments described herein are only illustrative of the principles ofthe present invention and that various modifications may be implementedby those skilled in the art without departing from the scope and spiritof the invention. Those skilled in the art could implement various otherfeature combinations and additions without departing from the scope andspirit of the invention. For example, there are made embodiments thatcite a single mobile communication device being monitored and it shouldbe known that these embodiments can allow for multiple mobilecommunication devices being monitored. An example for which would be aparent who has two or more children that they wish to monitor mobilecommunications for.

As discussed throughout this document, various features and elements ofsome invention embodiments will be discussed and illustrated. It will beappreciated that such discussion is for the purposes of illustrationonly, and is not intended to limit the scope of the invention. As anexample, many invention embodiments will find utility when practicedwith portable phones such as cellular phones, smartphones, iPhones,Blackberry, Android, Windows phone or any mobile phone or mobile devicewith memory and processor running an operating system referred to hereinas “cell phone” or “cell phones.” Cell phone in the context of thisspecification also means a cellular phone with an operating system thatis capable of running a mobile app. A common term (or name) for thistype of phone is “smartphone”. Examples of smartphones currently on themarket include but are not limited to iPhone 5, Samsung Galaxy S5 andthe Motorola Droid RAZR M.

It will be appreciated, however, that the inventive technology is usefulwith a wide variety of mobile communications devices, and cell phoneswith operating systems (“smartphones”) are but one example. Such devicesinclude but are not limited to portable processor based devices such aslaptops, tablets, gaming devices, entertainment devices, music players,cameras, smart phones, dedicated devices, and others. Additionally,reference is made by way of illustration to texting and instantmessaging. These are but two examples of electronic communications thatdifferent embodiments of the invention will find utility with.Additionally, many applications are described in the context of a parentmonitoring a child. It will be appreciated that many other applicationsare likewise within the scope of the invention, with virtually anyapplication in which a first user desires to monitor some aspects ofcommunications occurring via a second user. It will be appreciated thatthe use of these terms herein is intended to illustrate specificembodiments only, and will be appreciated that many otherdevices/terms/technologies will be applicable.

While various embodiments of the present invention have been shown anddescribed, it should be understood that other modifications,substitutions, and alternatives are apparent to one of ordinary skill inthe art. Such modifications, substitutions, and alternatives can be madewithout departing from the spirit and scope of the invention, whichshould be determined from the appended claims.

Various features of the invention are set forth in the appended claims.

What is claimed is:
 1. A method for monitoring mobile communication andgenerating alerts associated with targeted content, the methodcomprising: establishing a link between a monitoring user's mobilecommunication device and a monitored user's mobile communication deviceto receive communications for monitoring purposes; the monitored user'smobile communication device forwarding an incoming or outgoingcommunication to the monitoring user's mobile communication device; themonitored user's or monitoring user's mobile communication devicegenerating an alert if the incoming or outgoing communication on themonitored user's mobile communication device contains targeted content;and the monitoring user's mobile communication device providing thealert indicating that the incoming or outgoing communication containstargeted content; wherein the monitored user's mobile communicationdevice receives or is transmitting an electronic message comprising oneor more of a text message, message, app message, image, audio recording,audio clip, video, video clip, SMS or MMS message; and the monitoreduser's or monitoring user's mobile communication device comparing theelectronic message to i) a list of predefined text strings, images,audio clips, and/or video clips previously determined to be targetedcontent or ii) to an artificial intelligence (AI) model that is trainedon predefined text strings, images, audio clips, and/or video clipspreviously determined to be targeted content and that provides a ratingor scoring for identifying similarities to target content; andpredicated on a match or a comparable match, the monitored user's mobilecommunication device transmitting an alert to the monitoring user'scommunication device notifying that potential inappropriate or targetedcontent was detected or the monitoring user's mobile communicationdevice generating an alert notifying that potential inappropriate ortargeted content was detected.
 2. The method of claim 1, wherein amonitoring user's mobile communication device is authenticated toreceive communications from the monitored user's mobile communicationdevice for monitoring purposes.
 3. The method of claim 1, wherein theincoming or outgoing communication comprises at least one text messageor SMS message.
 4. The method of claim 1, wherein the incoming oroutgoing communication comprises at least one image or MMS message. 5.The method of claim 1, wherein the incoming or outgoing communicationcomprises at least one voice mail message.
 6. The method of claim 1,wherein the incoming or outgoing communication comprises at least onelive voice communication session or excerpt of a live voicecommunication session.
 7. The method of claim 1, wherein the incoming oroutgoing communication comprises at least one video clip.
 8. The methodof claim 1 further comprising: the monitoring user's mobilecommunication device further providing a description of the targetedcontent on the monitoring user's mobile communication device for reviewby the monitoring user.
 9. The method of claim 1, wherein the incomingor outgoing communication comprises at least one electronic message. 10.The method of claim 1, wherein the outgoing communications transmitted(forwarded) from the monitored user's mobile communication device to themonitoring user's mobile communication device are implemented in apeer-to-peer network architecture (peer-to-peer network scheme).
 11. Themethod of claim 1, wherein the forwarded communications from themonitored user's mobile communication device to the monitoring user'smobile communication device are exclusively outgoing (outbound)communications from the monitored user to one of more third parties. 12.The method of claim 1, wherein forwarded communications from themonitored user's mobile communication device to the monitoring user'smobile communication device are exclusively incoming (inbound)communications to the monitored user from one of more third parties. 13.The method of claim 1, wherein, predicated on a match or a comparablematch, the monitored user's mobile communication device forwarding(transmitting) the electronic message to the monitoring user's mobilecommunication device.
 14. The method of claim 1, wherein the incoming oroutgoing communication resides on the monitored user's mobilecommunication device; wherein the monitored user's mobile communicationdevice presents at least a portion of a communication on an outputinterface of the monitored user's mobile communication device; andwherein the monitored user's mobile communication device reads intomemory the presented portion of the communication in response to suchpresenting.
 15. The method of claim 14, wherein the output interfacecomprises a display of the monitored user's mobile communication device;wherein said presenting comprises displaying the at least a portion ofthe communication on the display; and wherein said reading into memorycomprises performing a screen capture or screen recording of at least aportion of the display.
 16. The method of claim 15, wherein the outputinterface comprises an audio output of the monitored user's mobilecommunication device; wherein said presenting comprises playing the atleast a portion of the communication on the audio output; and whereinsaid reading into memory comprises performing an audio recording oraudio capture of the portion of the communication.
 17. The method ofclaim 15, wherein the monitored user's communication device reads intomemory the presented portion of the communication at any time that theportion is presented on the output interface.
 18. The method of claim 1,wherein, if it is determined that the communication contains thetargeted content, the monitored user's mobile communication deviceobfuscates at least a portion of the targeted content prior to themonitored user's mobile communication forwarding the communication tothe monitoring user's mobile communication device.
 19. The method ofclaim 1, wherein, if it is determined that the communication containsthe targeted content, the monitoring user's mobile communication deviceobfuscating at least a portion of the targeted content after themonitoring user's mobile communication device receives the communicationfrom the monitored user's mobile communication device.
 20. The method ofclaim 18, wherein the obfuscating comprises blurring the image, video,video clip, video frame, MMS or electronic message.
 21. The method ofclaim 19, wherein the obfuscating comprises blurring the image, video,video clip, video frame, MMS or electronic message.
 22. The method ofclaim 18, wherein the obfuscating comprises audibly distorting at leasta portion of the audio clip.
 23. The method of claim 18, wherein theobfuscated communication is transmitted from the monitored user's mobilecommunication device to the monitoring user's mobile communicationdevice in a peer-to-peer network architecture (scheme) or a hybridpeer-to-peer network architecture (scheme).
 24. The method of claim 19,wherein before the obfuscating the communication is transmitted from themonitored user's mobile communication device to the monitoring user'smobile communication device in a peer-to-peer network architecture(scheme) or a hybrid peer-to-peer network architecture (scheme).
 25. Themethod of claim 18, wherein the obfuscated communication is transmittedvia a cloud computing scheme for storage, analysis, processing and/ortransmittal to the monitoring user's mobile communication device or themonitoring user's mobile communication device receives or retrieves thecommunication via a cloud computing scheme for obfuscating by themonitoring user's mobile communication device.
 26. The method of claim19, wherein before the obfuscating the communication is transmitted viaa cloud computing scheme for storage, analysis, processing and/ortransmittal to the monitoring user's mobile communication device or themonitoring user's mobile communication device receives or retrieves thecommunication via a cloud computing scheme for obfuscating by themonitoring user's mobile communication device.
 27. The method of claim18, wherein the obfuscated communication is stored on a server forstorage, analysis, processing and/or transmittal to the monitoringuser's mobile communication device or the monitoring user's mobilecommunication device receives or retrieves the communication from aserver for obfuscating by the monitoring user's mobile communicationdevice.
 28. The method of claim 19, wherein before the obfuscating thecommunication is stored on a server for storage, analysis, processingand/or transmittal to the monitoring user's mobile communication deviceor the monitoring user's mobile communication device receives orretrieves the communication from a server for obfuscating by themonitoring user's mobile communication device.
 29. The method of claim1, further comprising: determining whether the incoming or outgoingcommunication on the monitored user's mobile communication devicecontains targeted content; wherein said determining employs anartificial intelligence model to analyze the incoming or outgoingcommunication.
 30. The method of claim 29, wherein the determining isimplemented on the monitored user's mobile communication device.
 31. Themethod of claim 29, wherein the determining is implemented using acloud-based device.
 32. The method of claim 29, wherein the determiningis implemented on a server.
 33. The method of claim 29, wherein saiddetermining comprises inputting at least a portion of the incoming oroutgoing communication to the artificial intelligence model andproviding or receiving an output indicating whether the incoming oroutgoing communication on the monitored user's mobile communicationdevice contains targeted content.
 34. The method of claim 33, whereinthe artificial intelligence model comprises a neural network model thatis trained to detect targeted content.
 35. The method of claim 34,wherein the neural network model comprises a convolutional neuralnetwork (CNN) model, and wherein the output comprises a prediction. 36.The method of claim 1, wherein the forwarding of the communication fromthe monitored user's mobile communication device to the monitoringuser's mobile communication device is implemented in a hybridpeer-to-peer network architecture; wherein the hybrid peer-to-peerarchitecture comprises at least one peer-to-peer network model and atleast one client/server network model), and incorporates bidirectionalcommunications.
 37. The method of claim 1, wherein the communication istransmitted from the monitored mobile communication device to themonitoring mobile communication device based on one or more of: thecommunication being determined to meet one or more predefined criteria;a communication being determined to contain targeted content; or acommunication being determined to contain trained targeted content usingan artificial intelligence model.
 38. The method of claim 37, whereinthe communication that is determined to best meet the predefinedcriteria, contain targeted content, and/or contain trained targetedcontent is transmitted to the monitoring user's mobile communicationdevice at a set or random time interval.
 39. The method of claim 37,wherein the communication that is determined to best meet the predefinedcriteria, contain targeted content, and/or contain trained targetedcontent is transmitted on-demand by the user of the monitoring mobilecommunication device to the monitoring user's mobile communicationdevice.
 40. The method of claim 37, wherein at least a portion of thecommunication is obfuscated prior to being transmitted to the monitoringmobile communication device.
 41. The method in claim 37, wherein thecommunication is transmitted from the monitored mobile communicationdevice to the monitoring mobile communication device in a peer-to-peernetwork architecture or hybrid peer-to-peer network architecture. 42.The method in claim 37, wherein the communication is transmitted fromthe monitored mobile communication device to the monitoring mobilecommunication device via the cloud.
 43. The method in claim 37 whereinthe communication is transmitted from the monitored mobile communicationdevice to the monitoring mobile communication device via a server.
 44. Amethod for monitoring mobile communication and generating alertsassociated with targeted content, the method comprising: a monitoreduser's mobile communication device forwarding an incoming or outgoingcommunication for storage and analysis; the communication beingtransmitted via a cloud computing scheme to a monitoring user's mobilecommunication device or the monitoring user's mobile communicationdevice retrieving the communication via the cloud computing scheme; themonitoring user's mobile communication device generating an alert if,based on said analysis, the incoming or outgoing communication on themonitored user's mobile communication device contains targeted content;and the monitoring user's mobile communication device providing thealert indicating that the incoming or outgoing communication containstargeted content; wherein the monitored user's mobile communicationdevice receives or is transmitting an electronic message comprising oneor more of a text message, message, app message, image, audio recording,audio clip, video, video clip, SMS or MMS message; and the monitoreduser's or monitoring user's mobile communication device comparing theelectronic message to i) a list of predefined text strings, images,audio clips, and/or video clips previously determined to be targetedcontent or ii) to an artificial intelligence (AI) model that is trainedon predefined text strings, images, audio clips, and/or video clipspreviously determined to be targeted content and that provides a ratingor scoring for identifying similarities to target content; andpredicated on a match or a comparable match, the monitored user's mobilecommunication device transmitting an alert to the monitoring user'scommunication device notifying that potential inappropriate or targetedcontent was detected or the monitoring user's mobile communicationdevice generating an alert notifying that potential inappropriate ortargeted content was detected.
 45. The method of claim 44, wherein thecommunication being stored comprises at least one non-SMS text messageor SMS message.
 46. The method of claim 44, wherein the communicationbeing stored comprises at least one image or MMS message.
 47. The methodof claim 44, wherein the communication being stored comprises at leastone voice mail message.
 48. The method of claim 44, wherein thecommunication being stored comprises at least one live voicecommunication session or excerpt of a live voice communication session.49. The method of claim 44, wherein the communication being storedcomprises at least one video.
 50. The method of claim 44 furthercomprising: the monitoring user's mobile communication device furtherstoring a description of the targeted content for transmittal to orretrieval by the monitoring user's mobile communication device forreview by the monitoring user.
 51. The method of claim 44, wherein theincoming or outgoing communication comprises at least one electronicmessage.
 52. The method of claim 44, wherein the forwardedcommunications from the monitored user's mobile communication device areexclusively outgoing (outbound) communications from one or more thirdparties for transmittal to or retrieval by the monitoring user's mobilecommunication device.
 53. The method of claim 44, wherein the forwardedcommunications from the monitored user's mobile communication device areexclusively incoming (inbound) communications from one or more thirdparties for transmittal to or retrieval by the monitoring user's mobilecommunication device.
 54. The method of claim 44, the monitored user'smobile communication device storing the electronic message; and saidanalyzing associated with the stored electronic message is performed viacloud computing technologies.
 55. The method of claim 44, wherein saidanalyzing comprises comparing the stored electronic message to a list ofpredefined text strings, images, audio clips, or video clips previouslydetermined to be targeted content; and predicated on a match or acomparable match, the electronic message is transmitted to or retrievedby the monitoring user's mobile communication device.
 56. The method ofclaim 44, wherein the monitored user's mobile communication devicereceives or transmits an electronic message comprising one or more oftext messages, images, audio clips, video clips, SMS or MMS messages;the monitored user's mobile communication device storing the electronicmessage; and said analyzing associated with the stored electronicmessage is performed via cloud computing technologies.
 57. The method ofclaim 44, wherein said analyzing comprises comparing the storedelectronic message to a list of predefined text strings, images, audioclips, or video clips previously determined to be targeted content; andpredicated on a match or a comparable match, an alert is transmitted toor retrieved by the monitoring user's mobile communication devicenotifying that potential inappropriate or targeted content was detectedin the electronic message.
 58. The method of claim 44 furthercomprising: establishing a link via the cloud computing scheme betweenthe monitoring user's mobile communication device and the monitoreduser's mobile communication device to receive communications formonitoring purposes.